/[thuban]/branches/WIP-pyshapelib-bramz/test/test_save.py
ViewVC logotype

Annotation of /branches/WIP-pyshapelib-bramz/test/test_save.py

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1268 - (hide annotations)
Fri Jun 20 16:10:12 2003 UTC (21 years, 8 months ago) by bh
Original Path: trunk/thuban/test/test_save.py
File MIME type: text/x-python
File size: 20232 byte(s)
* Resources/XML/thuban-0.8.dtd: New DTD for the new file format
version.

* Thuban/Model/save.py (sort_data_stores): New. Make topological
sort of the data sources so they can be written with sources that
data sources that depend on other data sources come after the
sources they depend on.
(SessionSaver.__init__): Add idmap instance variable to map from
objects to the ids used in the file.
(SessionSaver.get_id, SessionSaver.define_id)
(SessionSaver.has_id): New. Methods to manage the idmap
(SessionSaver.write): Use thuban-0.8.dtd
(SessionSaver.write_session): Add a namespace on the session and
write out the data sources before the maps.
(SessionSaver.write_data_containers): New. Write the data
containers.
(SessionSaver.write_layer): Layer elements now refer to a
shapestore and don't contain filenames anymore.

* Thuban/Model/load.py (LoadError): Exception class to raise when
errors in the files are discovered
(SessionLoader.__init__): Define dispatchers for elements with a
thuban-0.8 namespace too.
(SessionLoader.check_attrs): New helper method to check and
convert attributes
(AttrDesc): New. Helper class for SessionLoader.check_attrs
(SessionLoader.start_fileshapesource)
(SessionLoader.start_derivedshapesource)
(SessionLoader.start_filetable, SessionLoader.start_jointable):
Handlers for the new elements in the new fileformat
(SessionLoader.start_layer): Handle the shapestore attribute in
addition to filename.
(SessionLoader.start_table, SessionLoader.end_table): Removed.
They were never used in the old formats and aren't needed for the
new.

* Thuban/Model/session.py (Session.DataContainers): New method to
return all "data containers", i.e. shapestores and tables

* test/xmlsupport.py (SaxEventLister.__init__)
(SaxEventLister.startElementNS, sax_eventlist): Add support to
normalize IDs.

* test/test_xmlsupport.py
(TestEventList.test_even_list_id_normalization): New test case for
id normalization

* test/test_load.py (LoadSessionTest.check_format): Use ID
normalization
(LoadSessionTest.thubanids, LoadSessionTest.thubanidrefs): New
class atrributes used for ID normalization
(TestSingleLayer, TestLayerVisibility, TestLabels.test)
(TestLayerProjection.test, TestRasterLayer.test): Adapt to new
file format
(TestJoinedTable): New test for loading sessions with joined
tables.
(TestLoadError): New. Test whether missing required attributes
cause a LoadError.

* test/test_save.py (SaveSessionTest.thubanids)
(SaveSessionTest.thubanidrefs): New class attributes for ID
normalization in .thuban files.
(SaveSessionTest.compare_xml): Use id-normalization.
(SaveSessionTest.testEmptySession)
(SaveSessionTest.testLayerProjection)
(SaveSessionTest.testRasterLayer)
(SaveSessionTest.testClassifiedLayer): Adapt to new file format.
(SaveSessionTest.testLayerProjection): The filename used was the
same as for testSingleLayer. Use a different one.
(SaveSessionTest.test_dbf_table)
(SaveSessionTest.test_joined_table): New test cases for saving the
new data sources structures.
(TestStoreSort, MockDataStore): Classes to test the sorting of the
data stores for writing.

1 bh 723 # Copyright (c) 2002, 2003 by Intevation GmbH
2 bh 292 # Authors:
3     # Bernhard Herzog <[email protected]>
4     #
5     # This program is free software under the GPL (>=v2)
6     # Read the file COPYING coming with Thuban for details.
7    
8     """
9     Test saving a thuban session as XML
10     """
11    
12     __version__ = "$Revision$"
13     # $Source$
14     # $Id$
15    
16     import os
17     import unittest
18     from StringIO import StringIO
19    
20     import xml.sax
21     import xml.sax.handler
22 jonathan 530 from xml.sax import make_parser, ErrorHandler, SAXNotRecognizedException
23 bh 292
24 bh 1245 import xmlsupport
25    
26 bh 292 import support
27     support.initthuban()
28    
29 bh 1268 import dbflib
30    
31     from Thuban.Lib.fileutil import relative_filename
32     from Thuban.Model.save import XMLWriter, save_session, sort_data_stores
33 bh 292 from Thuban.Model.session import Session
34     from Thuban.Model.map import Map
35 jonathan 947 from Thuban.Model.layer import Layer, RasterLayer
36 bh 292 from Thuban.Model.proj import Projection
37 bh 1268 from Thuban.Model.table import DBFTable
38     from Thuban.Model.transientdb import TransientJoinedTable
39     from Thuban.Model.data import DerivedShapeStore
40 bh 292
41 jonathan 1168 from Thuban.Model.classification import ClassGroupSingleton, ClassGroupRange, \
42     ClassGroupProperties
43 bh 292
44 jonathan 1168 from Thuban.Model.range import Range
45    
46    
47     class XMLWriterTest(unittest.TestCase):
48    
49     def testEncode(self):
50     """Test XMLWriter.encode"""
51     writer = XMLWriter()
52 jonathan 1200 eq = self.assertEquals
53 jonathan 1168
54 jonathan 1200 eq(writer.encode("hello world"), "hello world")
55     eq(writer.encode(unicode("hello world")), unicode("hello world"))
56 jonathan 1173
57 jonathan 1200 eq(writer.encode("\x80\x90\xc2\x100"),
58     "\xc2\x80\xc2\x90\xc3\x82\x100")
59     eq(writer.encode(u"\x80\x90\xc2\x100"),
60     "\xc2\x80\xc2\x90\xc3\x82\x100")
61     eq(writer.encode(u"\xFF5E"), "\xc3\xbf5E")
62 jonathan 1173
63 jonathan 1200 eq(writer.encode('&"\'<>'), "&amp;&quot;&apos;&lt;&gt;")
64     eq(writer.encode(unicode('&"\'<>')), "&amp;&quot;&apos;&lt;&gt;")
65 jonathan 1168
66 bh 1245 class SaveSessionTest(unittest.TestCase, support.FileTestMixin,
67     xmlsupport.ValidationTest):
68 bh 292
69 bh 1268 dtd = "http://thuban.intevation.org/dtds/thuban-0.8.dtd"
70     thubanids = [((dtd, n), (None, "id")) for n in
71     ["fileshapesource", "filetable", "jointable",
72     "derivedshapesource"]]
73     thubanidrefs = [((dtd, n), (None, m)) for n, m in
74     [("layer", "shapestore"),
75     ("jointable", "left"),
76     ("jointable", "right"),
77     ("derivedshapesource", "table"),
78     ("derivedshapesource", "shapesource")]]
79     del n, m, dtd
80    
81 bh 292 def compare_xml(self, xml1, xml2):
82 bh 1268 if 0:
83     for a, b in zip(sax_eventlist(xml1, self.thubanids,
84     self.thubanidrefs),
85     sax_eventlist(xml2, self.thubanids,
86     self.thubanidrefs)):
87     print a == b and ' ' or '*****'
88     print a
89     print b
90     self.assertEquals(xmlsupport.sax_eventlist(xml1, ids = self.thubanids,
91     idrefs = self.thubanidrefs),
92     xmlsupport.sax_eventlist(xml2, ids = self.thubanids,
93     idrefs = self.thubanidrefs))
94 bh 292
95     def testEmptySession(self):
96     """Save an empty session"""
97     session = Session("empty session")
98     filename = self.temp_file_name("save_emptysession.thuban")
99     save_session(session, filename)
100     session.Destroy()
101    
102     file = open(filename)
103     written_contents = file.read()
104     file.close()
105     self.compare_xml(written_contents,
106     '<?xml version="1.0" encoding="UTF-8"?>\n'
107 bh 1268 '<!DOCTYPE session SYSTEM "thuban-0.8.dtd">\n'
108     '<session title="empty session" '
109     'xmlns="http://thuban.intevation.org/dtds/thuban-0.8.dtd">'
110     '\n</session>\n')
111 bh 292
112 bh 1245 self.validate_data(written_contents)
113    
114 bh 292 def testSingleLayer(self):
115     """Save a session with a single map with a single layer"""
116     # deliberately put an apersand in the title :)
117     session = Session("single map&layer")
118     proj = Projection(["zone=26", "proj=utm", "ellps=clrk66"])
119     map = Map("Test Map", projection = proj)
120     session.AddMap(map)
121     # use shapefile from the example data
122     shpfile = os.path.join(os.path.dirname(__file__),
123     os.pardir, "Data", "iceland", "political.shp")
124 bh 723 layer = Layer("My Layer", session.OpenShapefile(shpfile))
125 bh 292 map.AddLayer(layer)
126    
127     filename = self.temp_file_name("save_singlemap.thuban")
128     save_session(session, filename)
129    
130     file = open(filename)
131     written_contents = file.read()
132     file.close()
133 jonathan 775 expected_template = '''<?xml version="1.0" encoding="UTF-8"?>
134 bh 1268 <!DOCTYPE session SYSTEM "thuban-0.8.dtd">
135     <session title="single map&amp;layer"
136     xmlns="http://thuban.intevation.org/dtds/thuban-0.8.dtd">
137     <fileshapesource id="D1" filename="%s" filetype="shapefile"/>
138 bh 292 <map title="Test Map">
139 jonathan 755 <projection name="Unknown">
140 bh 292 <parameter value="zone=26"/>
141     <parameter value="proj=utm"/>
142     <parameter value="ellps=clrk66"/>
143     </projection>
144 bh 1268 <layer title="My Layer" shapestore="D1"
145 jonathan 775 fill="None" stroke="#000000" stroke_width="1" visible="%s"/>
146 bh 292 </map>
147 bh 1268 </session>'''
148    
149 jonathan 775 expected_contents = expected_template % \
150     (os.path.join("..", "..", "Data", "iceland", "political.shp"),
151     "true")
152    
153 bh 292 self.compare_xml(written_contents, expected_contents)
154    
155 bh 1245 self.validate_data(written_contents)
156    
157 jonathan 775 layer.SetVisible(False)
158     save_session(session, filename)
159    
160     file = open(filename)
161     written_contents = file.read()
162     file.close()
163     expected_contents = expected_template % \
164     (os.path.join("..", "..", "Data", "iceland", "political.shp"),
165     "false")
166     self.compare_xml(written_contents, expected_contents)
167 bh 1245 self.validate_data(written_contents)
168 jonathan 775
169     session.Destroy()
170    
171 jonathan 755 def testLayerProjection(self):
172 bh 1268 """Test saving layers with projections"""
173 jonathan 755 # deliberately put an apersand in the title :)
174     session = Session("single map&layer")
175     proj = Projection(["zone=26", "proj=utm", "ellps=clrk66"])
176     map = Map("Test Map", projection = proj)
177     session.AddMap(map)
178     # use shapefile from the example data
179     shpfile = os.path.join(os.path.dirname(__file__),
180     os.pardir, "Data", "iceland", "political.shp")
181     layer = Layer("My Layer", session.OpenShapefile(shpfile))
182     proj = Projection(["proj=lcc", "ellps=clrk66"], "Layer Projection")
183     layer.SetProjection(proj)
184     map.AddLayer(layer)
185 bh 292
186 bh 1268 filename = self.temp_file_name("save_layerproj.thuban")
187 jonathan 755 save_session(session, filename)
188     session.Destroy()
189 bh 292
190 jonathan 755 file = open(filename)
191     written_contents = file.read()
192     file.close()
193     expected_contents = '''<?xml version="1.0" encoding="UTF-8"?>
194 bh 1268 <!DOCTYPE session SYSTEM "thuban-0.8.dtd">
195     <session title="single map&amp;layer"
196     xmlns="http://thuban.intevation.org/dtds/thuban-0.8.dtd">
197     <fileshapesource id="D1" filename="%s" filetype="shapefile"/>
198 jonathan 755 <map title="Test Map">
199     <projection name="Unknown">
200     <parameter value="zone=26"/>
201     <parameter value="proj=utm"/>
202     <parameter value="ellps=clrk66"/>
203     </projection>
204 bh 1268 <layer title="My Layer" shapestore="D1"
205 jonathan 775 fill="None" stroke="#000000" stroke_width="1" visible="true">
206 jonathan 755 <projection name="Layer Projection">
207     <parameter value="proj=lcc"/>
208     <parameter value="ellps=clrk66"/>
209     </projection>
210     </layer>
211     </map>
212     </session>''' % os.path.join("..", "..", "Data", "iceland",
213     "political.shp")
214     #print written_contents
215     #print "********************************************"
216     #print expected_contents
217     self.compare_xml(written_contents, expected_contents)
218    
219 bh 1245 self.validate_data(written_contents)
220    
221 jonathan 947 def testRasterLayer(self):
222     # deliberately put an apersand in the title :)
223     session = Session("single map&layer")
224     map = Map("Test Map")
225     session.AddMap(map)
226     # use shapefile from the example data
227     imgfile = os.path.join(os.path.dirname(__file__),
228     os.pardir, "Data", "iceland", "island.tif")
229     layer = RasterLayer("My RasterLayer", imgfile)
230     map.AddLayer(layer)
231 bh 1245
232 jonathan 947 filename = self.temp_file_name("save_singlemap.thuban")
233     save_session(session, filename)
234     session.Destroy()
235 bh 1245
236 jonathan 947 file = open(filename)
237     written_contents = file.read()
238     file.close()
239     expected_contents = '''<?xml version="1.0" encoding="UTF-8"?>
240 bh 1268 <!DOCTYPE session SYSTEM "thuban-0.8.dtd">
241     <session title="single map&amp;layer"
242     xmlns="http://thuban.intevation.org/dtds/thuban-0.8.dtd">
243 jonathan 947 <map title="Test Map">
244 bh 1268 <rasterlayer title="My RasterLayer" filename="%s"
245 jonathan 947 visible="true">
246     </rasterlayer>
247     </map>
248 bh 1268 </session>''' % os.path.join(os.path.dirname(__file__),
249 jonathan 947 os.pardir, "Data", "iceland",
250     "island.tif")
251     #print written_contents
252     #print "********************************************"
253     #print expected_contents
254     self.compare_xml(written_contents, expected_contents)
255 jonathan 755
256 bh 1245 self.validate_data(written_contents)
257    
258 jonathan 1168 def testClassifiedLayer(self):
259     """Save a session with a single map with a single layer
260     with a classificaton.
261     """
262     # deliberately put an apersand in the title :)
263     session = Session("single map&layer")
264     proj = Projection(["zone=26", "proj=utm", "ellps=clrk66"])
265     map = Map("Test Map", projection = proj)
266     session.AddMap(map)
267     # use shapefile from the example data
268     shpfile = os.path.join(os.path.dirname(__file__),
269     os.pardir, "Data", "iceland", "political.shp")
270     layer = Layer("My Layer", session.OpenShapefile(shpfile))
271     map.AddLayer(layer)
272 jonathan 755
273 jonathan 1168 clazz = layer.GetClassification()
274    
275     clazz.SetField("AREA")
276    
277     clazz.AppendGroup(ClassGroupSingleton(42,
278     ClassGroupProperties(),
279     "single"))
280     clazz.AppendGroup(ClassGroupSingleton("text",
281     ClassGroupProperties(),
282     "single-text"))
283    
284     clazz.AppendGroup(ClassGroupRange(0, 42,
285     ClassGroupProperties(),
286     "range"))
287    
288     range = ClassGroupRange(Range("[0;42]"))
289     range.SetProperties(ClassGroupProperties())
290     range.SetLabel("new-range")
291     clazz.AppendGroup(range)
292    
293     filename = self.temp_file_name("save_singlemap.thuban")
294     save_session(session, filename)
295    
296     file = open(filename)
297     written_contents = file.read()
298     file.close()
299     expected_template = '''<?xml version="1.0" encoding="UTF-8"?>
300 bh 1268 <!DOCTYPE session SYSTEM "thuban-0.8.dtd">
301     <session title="single map&amp;layer"
302     xmlns="http://thuban.intevation.org/dtds/thuban-0.8.dtd">
303     <fileshapesource id="D1" filename="%s" filetype="shapefile"/>
304 jonathan 1168 <map title="Test Map">
305     <projection name="Unknown">
306     <parameter value="zone=26"/>
307     <parameter value="proj=utm"/>
308     <parameter value="ellps=clrk66"/>
309     </projection>
310 bh 1268 <layer title="My Layer" shapestore="D1"
311 jonathan 1168 fill="None" stroke="#000000" stroke_width="1" visible="%s">
312     <classification field="AREA" field_type="double">
313     <clnull label="">
314     <cldata fill="None" stroke="#000000" stroke_width="1"/>
315     </clnull>
316     <clpoint value="42" label="single">
317     <cldata fill="None" stroke="#000000" stroke_width="1"/>
318     </clpoint>
319     <clpoint value="text" label="single-text">
320     <cldata fill="None" stroke="#000000" stroke_width="1"/>
321     </clpoint>
322     <clrange range="[0;42[" label="range">
323     <cldata fill="None" stroke="#000000" stroke_width="1"/>
324     </clrange>
325     <clrange range="[0;42]" label="new-range">
326     <cldata fill="None" stroke="#000000" stroke_width="1"/>
327     </clrange>
328     </classification>
329     </layer>
330     </map>
331 bh 1245 </session>'''
332    
333 jonathan 1168 expected_contents = expected_template % \
334     (os.path.join("..", "..", "Data", "iceland", "political.shp"),
335     "true")
336    
337     #print written_contents
338     #print "********************************************"
339     #print expected_contents
340     self.compare_xml(written_contents, expected_contents)
341    
342 bh 1245 self.validate_data(written_contents)
343    
344 jonathan 1168 session.Destroy()
345    
346 bh 1268 def test_dbf_table(self):
347     """Test saving a session with a dbf table link"""
348     session = Session("a DBF Table session")
349     # use shapefile from the example data
350     dbffile = os.path.join(os.path.dirname(__file__),
351     os.pardir, "Data", "iceland", "political.dbf")
352     table = session.AddTable(DBFTable(dbffile))
353 jonathan 1168
354 bh 1268 filename = self.temp_file_name("save_singletable.thuban")
355     save_session(session, filename)
356    
357     file = open(filename)
358     written_contents = file.read()
359     file.close()
360     expected_template = '''<?xml version="1.0" encoding="UTF-8"?>
361     <!DOCTYPE session SYSTEM "thuban-0.8.dtd">
362     <session title="a DBF Table session"
363     xmlns="http://thuban.intevation.org/dtds/thuban-0.8.dtd">
364     <filetable id="D1" filename="%s" filetype="DBF" title="political"/>
365     </session>'''
366    
367     expected_contents = expected_template % dbffile
368     self.compare_xml(written_contents, expected_contents)
369    
370     def test_joined_table(self):
371     """Test saving a session with joined table"""
372     # Create a simple table to use in the join
373     dbffile = self.temp_file_name("save_joinedtable.dbf")
374     dbf = dbflib.create(dbffile)
375     dbf.add_field("RDTYPE", dbflib.FTInteger, 10, 0)
376     dbf.add_field("TEXT", dbflib.FTString, 10, 0)
377     dbf.write_record(0, {'RDTYPE': 8, "TEXT": "foo"})
378     dbf.write_record(1, {'RDTYPE': 2, "TEXT": "bar"})
379     dbf.write_record(2, {'RDTYPE': 3, "TEXT": "baz"})
380     dbf.close()
381    
382     # Create the session and a map
383     session = Session("A Joined Table session")
384     try:
385     map = Map("Test Map")
386     session.AddMap(map)
387    
388     # Add the dbf file to the session
389     dbftable = session.AddTable(DBFTable(dbffile))
390    
391     # Create a layer with the shapefile to use in the join
392     shpfile = os.path.join(os.path.abspath(os.path.dirname(__file__)),
393     os.pardir, "Data", "iceland",
394     "roads-line.shp")
395     layer = Layer("My Layer", session.OpenShapefile(shpfile))
396     map.AddLayer(layer)
397    
398     # Do the join
399     store = layer.ShapeStore()
400     #for col in store.Table().Columns():
401     # print col.name
402     joined = TransientJoinedTable(session.TransientDB(),
403     store.Table(), "RDLNTYPE",
404     dbftable, "RDTYPE")
405     store = session.AddShapeStore(DerivedShapeStore(store, joined))
406     layer.SetShapeStore(store)
407    
408     # Save the session
409     filename = self.temp_file_name("save_joinedtable.thuban")
410     save_session(session, filename)
411    
412     # Read it back and compare
413     file = open(filename)
414     written_contents = file.read()
415     file.close()
416     expected_template = '''<?xml version="1.0" encoding="UTF-8"?>
417     <!DOCTYPE session SYSTEM "thuban-0.8.dtd">
418     <session title="A Joined Table session"
419     xmlns="http://thuban.intevation.org/dtds/thuban-0.8.dtd">
420     <fileshapesource filename="%(shpfile)s"
421     filetype="shapefile" id="D142197204"/>
422     <filetable filename="%(dbffile)s"
423     title="save_joinedtable"
424     filetype="DBF" id="D141881756"/>
425     <jointable id="D142180284"
426     title="Join of roads-line and save_joinedtable"
427     leftcolumn="RDLNTYPE" left="D142197204"
428     rightcolumn="RDTYPE" right="D141881756"/>
429     <derivedshapesource id="D141915644"
430     table="D142180284"
431     shapesource="D142197204"/>
432     <map title="Test Map">
433     <layer title="My Layer"
434     shapestore="D141915644" visible="true"
435     stroke="#000000" stroke_width="1" fill="None"/>
436     </map>
437     </session>'''
438    
439     expected_contents = expected_template % {
440     "dbffile": relative_filename(self.temp_dir(), dbffile),
441     "shpfile": relative_filename(self.temp_dir(), shpfile)
442     }
443     self.compare_xml(written_contents, expected_contents)
444     finally:
445     session.Destroy()
446     session = None
447    
448    
449     class MockDataStore:
450    
451     """A very simple data store that only has dependencies"""
452    
453     def __init__(self, name, *dependencies):
454     self.name = name
455     self.dependencies = dependencies
456    
457     def __repr__(self):
458     return self.name
459    
460     def Dependencies(self):
461     return self.dependencies
462    
463    
464     class TestStoreSort(unittest.TestCase):
465    
466     def check_sort(self, containers, sorted):
467     """Check whether the list of data containers is sorted"""
468     # check whether sorted is in the right order
469     seen = {}
470     for container in sorted:
471     self.failIf(id(container) in seen,
472     "Container %r at least twice in %r" % (container,
473     sorted))
474     for dep in container.Dependencies():
475     self.assert_(id(dep) in seen,
476     "Dependency %r of %r not yet seen" % (dep,
477     container))
478     seen[id(container)] = 1
479     # check whether all of containers is in sorted
480     for container in containers:
481     self.assert_(id(container) in seen,
482     "Container %r in containers but not in sorted")
483     self.assertEquals(len(containers), len(sorted))
484    
485     def test_sort_data_stores(self):
486     """Test Thuban.Model.save.sort_data_stores"""
487     d1 = MockDataStore("d1")
488     d2 = MockDataStore("d2")
489     d3 = MockDataStore("d3", d1)
490     d4 = MockDataStore("d4", d1, d3)
491    
492     containers = [d4, d1, d2, d3]
493     self.check_sort(containers, sort_data_stores(containers))
494     containers = [d1, d3, d2, d4]
495     self.check_sort(containers, sort_data_stores(containers))
496    
497    
498    
499 bh 292 if __name__ == "__main__":
500     # Fake the __file__ global because it's needed by a test
501     import sys
502     __file__ = sys.argv[0]
503 bh 723 support.run_tests()

Properties

Name Value
svn:eol-style native
svn:keywords Author Date Id Revision

[email protected]
ViewVC Help
Powered by ViewVC 1.1.26