|DataLab is a compact statistics package aimed at exploratory data analysis. Please visit the DataLab Web site for more information....|
|Home Features of DataLab Loading and Storing Data Loading Data Importing Text Data Import Script Example|
|See also: Importing Text Data
Import Script Example
The following example shows how to import data from a text file. The dataset "dusttrak_20120920.txt" contains the measurements of fine particulate matter in the air, obtained during the visit of a cafe. The device to measure the dust (DustTrak DRX Aerosol Monitor, TSI Inc.) is an optical particle counter which is able to discriminate different particulate sizes. The device stores the data in a simple proprietary text format which contains various general informations in a header followed by the measured data. The length of the header may be varying, depending on the settings of the instrument and the notes added by the operator.
The data blocks starts two lines after a trigger line (indicated by red color in the sample text below). Each data point is stored in a separate line and contains the date and time of the measurement and five values for different fractions of the sampled particulate matter. All values in a line are separated by commas.
TrakPro Version 4.30 ASCII Data File Model: DustTrak DRX Model Number: 8534 Serial Number: 8534100401 Test ID: 001 Test Abbreviation: MANUAL_001 Start Date: 20.09.2012 Start Time: 11:56:48 Duration (dd:hh:mm:ss): 0:02:48:00 Log Interval (mm:ss): 00:30 Number of points: 337 Notes: Statistics Channel: PM1 PM2.5 RESP PM10 TOTAL Units: mg/m³ mg/m³ mg/m³ mg/m³ mg/m³ Average: 0.050 0.050 0.051 0.054 0.064 Minimum: 0.004 0.004 0.004 0.004 0.004 Time of Minimum: 12:00:18 12:00:18 12:00:48 12:00:48 12:00:48 Date of Minimum: 20.09.2012 20.09.2012 20.09.2012 20.09.2012 20.09.2012 Maximum: 0.154 0.155 0.156 0.169 0.367 Time of Maximum: 12:31:18 12:31:18 12:31:18 12:31:18 14:14:48 Date of Maximum: 20.09.2012 20.09.2012 20.09.2012 20.09.2012 20.09.2012 Calibration Sensor: AEROSOL Cal. date 28.06.2012 Date,Time,PM1,PM2.5,RESP,PM10,TOTAL dd.MM.yyyy,hh:mm:ss,mg/m³,mg/m³,mg/m³,mg/m³,mg/m³ 20.09.2012,11:57:18,0.005,0.005,0.005,0.006,0.010 20.09.2012,11:57:48,0.005,0.005,0.005,0.005,0.006 20.09.2012,11:58:18,0.005,0.005,0.005,0.006,0.011 20.09.2012,11:58:48,0.005,0.005,0.005,0.005,0.009 20.09.2012,11:59:18,0.005,0.005,0.005,0.006,0.009 20.09.2012,11:59:48,0.005,0.005,0.005,0.005,0.011 20.09.2012,12:00:18,0.004,0.004,0.005,0.005,0.007 20.09.2012,12:00:48,0.004,0.004,0.004,0.004,0.004 20.09.2012,12:01:18,0.004,0.005,0.005,0.005,0.005 ..... ..... .....
In order to scan the data set one first has to search for the trigger line, then skip the next line and start the analysis of the following lines. These data lines are analysed line by line using the following data extraction script:
pos(1) scandatetime ('dd.MM.yyyy,HH:mm:ss', DATE) emit (C1=DATE) copyuntil(',',DATESTR) emit(RowName=DATESTR); copyuntil(',',TIMESTR) assign(SEP=' ') emit(AppendRowName=SEP); emit(AppendRowName=TIMESTR); copyuntil(',',VALUE) emit (C2=VALUE) copyuntil(',',VALUE) emit (C3=VALUE) copyuntil(',',VALUE) emit (C4=VALUE) copyuntil(',',VALUE) emit (C5=VALUE) copyuntil(',',VALUE) emit (C6=VALUE)