Welcome to Sol 3 Sign in | Join | Help
CS Search | Live Search Search

Sol 3

Home of Barrows Software Solutions, LLC

Handling gigantic XML files

File Details
Downloads: 198 File Size: 59.1kB
Posted By: Keith Barrows Views: 192
Date Added: Fri, Dec 15 2006
My XML is not a standard XML layout (standard = parent, child, grandchild, etc data).  The client has taken all 1400 tables and called them generically "a category".  A category has a name (table name in this case) and a GUID.  Each category is broken into attributes (column names in this case).  Then the attributes are joined to the categories via a many-to-many join table.  So, we've condensed (screwed up) the relational layout into 3 tables.  This is dumped to a schema.xml with <ATTRIBUTES>, <CATEGORIES> & <CATEGORY_ATTRIBUTES> as the main sections.
 
The data is then exported from all 1400 tables into a single xml document that is comprised of "data point" nodes.  (I do not have an XSD of any sort so am waiting patiently for my system to load the 8+GB file).  I am assuming it is structured something like:
 
<NODE Category="guid" Attribute="guid" Value="......." />
 
There is probably more to it than that but let's go with this right now.
 
I now need to write an app that reads in the schema.xml file (10KB in size) and display the Table (category) list.  User selects a table and gets the column (attribute) list.  I then take that category/attribute and have to search the data.xml doc for all nodes with the same value.  The idea is to give a developer a look into the data, let them pick what the PK (unique value) column should be and see if there are dups.  If there are it is a bad data.xml file...
 
Any suggestions?  The logic is pretty straight forward.  I just want the user experience to be a little better than molasses in a cryogenic tank...

Comments
No comments exist for this file.

Add Comment

Name (required)
Web Site (optional)
Comment (required)
Add
CS Build: 2.1.61129.2
1999
Listed on the CS Listings Powered By Community Server Themed by nb development