SQL Import Part 2

Travian provides WorldMap data as a SQL dump; this is very simple and consists of rows of INSERT statements – At first glance this looks great and we now have many Travian Utilities using it.

When we import this into SQL the obvious starting point is to pass the INSERT statements to SQL and let it ‘do it’s thing’ – expect there are a few issues:-

1. Travian don’t use Microsoft SQL and part of the INSERT statement is incompatible (quote around table name)
2. Every INSERT statement for every Travian server tries to INSERT into the same table
3. Multiple INSERT statements are not efficient to import

So let’s take each of these issues:-

1. Simple, modify the INSERT statement. This means we now have to read all the INSERT statements, modify them, then pass them on.

2. Simple. As we have to modify every INSERT statement anyway for compatibility, this is a very fast in-memory operation to correct

2. For most Travian Utilities this is probably not a major issue, but for Travian iMailer this is a BIG issue and would mean our import of all 300+ servers, every day, could take hours. Instead we read all the INSERT statements, process them and bulk import into SQL