Load from CSV ignores last dataline

[Yusuf Zorlu]

We’re loading a lot of data from CSV-Files with  tkbmCSVStreamFormat. Today we noticed that files where the data-lines are in unix-format delimited with LF are not loaded completely into tkbmmemtable. But this is not only in unix format. If we convert the LF into CR+LF the same problem exists.

example:

“Auftragskonto”;”Buchungstag”
“DE 1″;”30.12.20”
“DE 2″;”30.12.20”
“DE 3″;”30.12.20”

if the line 4 (empty one after DE3) does not have a LF or CR then the line 3 with DE 3 will not be loaded. I’ve tried this with 7.92.0 and 7.91.1

[kimbomadsen]

Hi,

In latest kbmMemTable version, open kbmMemCSVStreamFormat.pas, locate line 904 and change that section to look like this:

...
          // Check if need another chunk.
          if FBufidx>=FBufSize then
          begin
               if bEOF then
                  break;
               DoGetChunk;
               continue;
          end
          else
...

Then please check again if the problem has been solved and report back.

Kim/C4D

[Yusuf Zorlu]

Your change solved the problem.

[bobmc998]

Hi Kim,

I’m using version 7.97.00 and I have the same issue as above: when LF is used as the end of line character, the last line of the csv file is not imported. When CR+LF is used, it works fine.

I checked the source of the kbmMemCSVStreamFormat file and the code is as shown above.

Thanks,

Didier

[kimbomadsen]

Hi,
Is it with any CSV file of any length? Can you please try to send an extremely simple sample for me to test?

/Kim

[Author]

Posts