ID #1181

Data get truncated when importing a large CSV-file.


SQLyog is is able to import data CSV files to MySQL using two different techniques:
1) using the Microsoft ODBC Text driver (that ships with every windows verson) for connecting to the file.
2) using MySQL "LOAD DATA LOCAL INFILE ..." syntax


The ODBC Text driver has limitations with very large data sets/files:
1) Tables are limited to a maximum of 255 fields/columns.
2) Field/column names are limited to 64 characters, and field/column widths are limited to 32,766 characters.
3) Records/rows are limited to 65,000 bytes in total.


Please refer to: msdn.microsoft.com/en-us/library/ms716438%28VS.85%29.aspx


If your file has data exceeding either of the above-mentioned limitations we cannot import all data from the file as we simply do not receive all the data from the ODBC-driver.


Consider the options instead then:
1) Avoid CSV-files. Try to import directly from the application who generated the CSV-files instead. It is possible if an ODBC-driver for that application (or its datastore) exists.
2) Import data using LOAD DATA syntax. The disadvantage with this option is that the table structure must be created manually in advance.



Categories for this entry

Tags: -

Related entries:

You can comment this FAQ