` Printed Icetips Article

Icetips Article

Par2: Process ASCII file using DOS driver
1998-04-07 -- John Hickey

Originally, I used the ASCII driver and it was taking a long time to import 1000 records.
The problem was not the adding of the records, but the reading of the ASCII file!

I decided to use the DOS driver to read the ASCII file in big chunks, and
then parse it out.  I defined a DOS file (DOSFILE) with one field (LINE)
that was 20000 long.  I then used the following code to parse it out:

NN#=0
SET(DOSFILE,1)
LOOP
 Next(DOSFILE)               !Reads in 20000 characters
 IF ERROR() Then BREAK.
 LOOP                              !Begin Parse
   X#=instring('<13,10>',clip(DOS:Line),1,1)   !Looks for Carriage Return/Line Feed
   IF ~X#
       ASC:Line=DOS:Line     !If not found, it's a partial line
       nn#=1                           !set a flag and
       Break                            !read in next DOS Line
   Else
      If ~nn#                           !if not a partial line
        ASC:Line=''                  !re-set ASC:Line
      End
   End
   nn#=0                     !re-set partial line flag
   ASC:Line=clip(ASC:Line) & sub(DOS:Line,1,x#-1)  !This is the actual ASC record now
   DOS:Line=sub(DOS:Line,x#+2,20000)   !shortens the DOS line for next parse
   !
   !
   !
 END              !End Parse
END             !End DOS loop

I put this into a Processing Template and it works great.  If you do that,
you don't need the first five lines (or the last END); the processing
template takes care of that for you.  You only need the lines from "Begin
Parse" to "End Parse".

The increase in speed was quite dramatic!

Printed November 21, 2024, 6:58 am
This article has been viewed/printed 35342 times.