COPY-LOB vs IMPORT UNFORMATTED

Hello All,

I am confused about what should I choose between copy-lob and import unformatted. I created one dummy .txt file (with 50 lines and 3 columns) and tried reading file with both methods in multiple iterations.

COPY-LOB seems slightly faster than IMPROT UNFORMATTED. Please suggest if we should straight away choose COPY-LOB over IMPORT UNFORMATTED or there are some conditional parameters for the same.

Please suggest.

Regards
 

Cringer

ProgressTalk.com Moderator
Staff member
I prefer COPY-LOB. I think it makes the code cleaner and it's easier to handle errors. But you start to run into problems when the file is large because the whole file is loaded into memory and if you don't have enough memory you will crash your process.
 

TomBascom

Curmudgeon
For reading and writing a LONGCHAR, COPY-LOB is much more convenient. I agree with James that it is cleaner coding too.

Stefan and James both point to problems working with large LONGCHAR data. But those issues are distinct from “which one is better to read it into a variable in the first place?” and addressing them requires knowledge of what you plan to do with that data after you have read it. I can confidently say that a file with 50 lines and 3 columns will not have any of those issues regardless of what you are going to do with it.

Worrying about small differences in performance with such a tiny test case is silly. Whatever difference you have measured is almost certainly overwhelmed by noise in the environment.
 

Stefan

Well-Known Member
For a tiny file Tom is entirely correct.
For anything else, any function whose performance degrades exponentially should be avoided (note that the function in this case is not copy-lob, but num-entries and entry).


On ABL Dojo the program is aborted for taking too long - note that the import file lines are small (three guids) - times in milliseconds:

Code:
Lines    Create   Import Copy-LOB
    50        1        0        0
   100        1        1        1
   200        2        0        3
   400        4        1        9
   800        8        2       35
  1600       14        4      134
  3200       28        7      532
  6400       56       14     2148

The results of a local run:

Code:
Lines    Create   Import Copy-LOB
    50        7        1        0
   100       14        1        1
   200       29        1        3
   400       60        2        6
   800      120        3       23
  1600      239        7       83
  3200      479       20      326
  6400      957       29     1297
 12800     1918       54     5605
 25600     3831      111    22888
 51200     7663      224    88837

So that's 89 seconds vs 0.2 seconds for a 51 thousand line file (old Excel used to have a 64k line limit - this was increased for a reason).
 

TomBascom

Curmudgeon
Stefan, could you explain what you are doing in that code? I don't grok what it is that you are testing and why you are showing columns for "import" vs "copy-lob". The two snippets being measured against each other don't seem at all equivalent to me.
 

TomBascom

Curmudgeon
Maybe you're comparing how to process a file one line at a time?

That's not an "apples to apples" comparison of IMPORT vs COPY-LOB though. That's two very different algorithms being applied to a problem that we don't know the OP cares about. The problem could be to just import text to store in a "notes" field for all that we know.
 

Stefan

Well-Known Member
Maybe you're comparing how to process a file one line at a time?

That's not an "apples to apples" comparison of IMPORT vs COPY-LOB though. That's two very different algorithms being applied to a problem that we don't know the OP cares about. The problem could be to just import text to store in a "notes" field for all that we know.

I am indeed actually doing "something" with the contents of the file per line. I am also going on the assumption that if someone is looking at copy-lob vs import that line processing is required, otherwise I do not see why import would even be considered in the first place.

I am only warning that if you need to handle the lines in the file to not be enticed by copy-lob just based on how cool and clean it looks on the outside.
 

TomBascom

Curmudgeon
I suspect that there are many legacy usages of IMPORT that could (should) be easily replaced with COPY-LOB. If you are learning about Progress by looking at legacy code you might wonder why people did not use COPY-LOB.

I agree -- if you are doing line by line processing (or something similar) then IMPORT is a much better choice.

OTOH if you are slurping a file into a LONGCHAR and not doing line by line (or similar repetitive string handling) then COPY-LOB is more appropriate.
 
I am indeed actually doing "something" with the contents of the file per line. I am also going on the assumption that if someone is looking at copy-lob vs import that line processing is required, otherwise I do not see why import would even be considered in the first place.

I am only warning that if you need to handle the lines in the file to not be enticed by copy-lob just based on how cool and clean it looks on the outside.

Thanks for this Great Explanation and Example Stefan !!!
 
I suspect that there are many legacy usages of IMPORT that could (should) be easily replaced with COPY-LOB. If you are learning about Progress by looking at legacy code you might wonder why people did not use COPY-LOB.

I agree -- if you are doing line by line processing (or something similar) then IMPORT is a much better choice.

OTOH if you are slurping a file into a LONGCHAR and not doing line by line (or similar repetitive string handling) then COPY-LOB is more appropriate.
Thanks for your Valuable inputs Tom !!!
 
Top