Parent Directory
| 
Revision Log
| Sticky Revision: |
made a copy
2006-05-15 Didrik Pinte <dpinte@itae.be> Ashamed ... still some tabs in the files * Thuban/version.py : tabs removed * Thuban/Model/transientdb.py : tabs removed
2006-05-15 Didrik Pinte <dpinte@itae.be> * Thuban/version.py : coding style correction * Thuban/Model/transientdb.py : coding style correction
2006-05-15 Didrik Pinte <dpinte@itae.be> * Thuban/version.py : Updated imports to support pysqlite2 * Thuban/Model/transientdb.py : Updated imports to support pysqlite2 Patched the querying system to support the param style of pysqlite2 --Cette ligne, M thuban/Thuban/Model/transientdb.py M thuban/Thuban/version.py M thuban/ChangeLog
Enable the experimental attribute editing again and introduce a command line switch to actually activate it * Thuban/UI/main.py (options): New. Container for options set on the commmand line (main): Add the --enable-attribute-editing flag. * Thuban/UI/identifyview.py (IdentifyView.__init__): If attribute editing is enabled use the grid ctrl which allows editing of the values * Thuban/Model/transientdb.py (AutoTransientTable.write_record): New. Just delegate this to the underlying table.
* Thuban/Model/table.py (OldTableInterfaceMixin): Removed. (DBFTable, MemoryTable): Do not derive from OldTableInterfaceMixin anymore * Thuban/Model/transientdb.py (TransientTableBase) (AutoTransientTable): Do not derive from OldTableInterfaceMixin anymore * test/test_table.py: Removed since the old interface it tests is gone.
Tweak the usage of the sqlite database to make common use cases more responsive. In most cases copying the data to the sqlite database takes so long that using sqlite doesn't have enough advantages. (TransientTableBase.ValueRange): Add comments about performance and query the min and max in separate statements because only that way will indexes be used. (TransientTableBase.UniqueValues): Add some comments about performance. (AutoTransientTable.ValueRange, AutoTransientTable.UniqueValues): Do not copy the data to the transient DB but use the transient copy if it exists. See the new comments for the performance trade offs
Make the table interface distinguish between row ids (an integer that uniquely identifies a row) and row ordinals (a simple row count from 0 to NumRows() - 1) * Thuban/Model/postgisdb.py (PostGISTable.RowIdToOrdinal) (PostGISTable.RowOrdinalToId): New methods to conver between row ids and row ordinals (PostGISTable.ReadRowAsDict, PostGISTable.ReadValue): New keyword parameter row_is_ordinal to indicate whether the row parameter is the row id or the ordinal * Thuban/Model/transientdb.py (TransientTableBase.RowIdToOrdinal) (TransientTableBase.RowOrdinalToId) (AutoTransientTable.RowIdToOrdinal) (AutoTransientTable.RowOrdinalToId): Same new methods as in PostGISTable. (TransientTableBase.ReadRowAsDict, TransientTableBase.ReadValue) (AutoTransientTable.ReadRowAsDict, AutoTransientTable.ReadValue): Same new parameter as in PostGISTable. * Thuban/Model/table.py (DBFTable.RowIdToOrdinal) (DBFTable.RowOrdinalToId, MemoryTable.RowIdToOrdinal) (MemoryTable.RowOrdinalToId): Same new methods as in PostGISTable. (DBFTable.ReadValue, DBFTable.ReadRowAsDict) (MemoryTable.ReadValue, MemoryTable.ReadRowAsDict): Same new parameter as in PostGISTable. * Thuban/UI/tableview.py (DataTable.RowIdToOrdinal) (DataTable.RowOrdinalToId): New methods to convert between row ids and row ordinals. (TableGrid.SelectRowById): New method to select a row based on its ID as opposed to its ordinal (DataTable.GetValue, TableGrid.OnRangeSelect) (TableGrid.OnSelectCell, LayerTableGrid.select_shapes) (QueryTableFrame.OnQuery, QueryTableFrame.get_selected) (LayerTableFrame.__init__): Convert between row ids and row ordinals as appropriate * test/postgissupport.py (PostGISDatabase.__init__): Add doc-string. (PostGISDatabase.initdb): The optional third item in a tuple in tables is now a (key, value) list with additional arguments to pass to upload_shapefile (upload_shapefile): New parameter gid_offset to allow gids that are not the same as the shapeids in the shapefile (PostgreSQLServer.get_default_static_data_db): Use the new gid_offset to make the gids in landmarks 1000 higher than the shapeids in the shapefile * test/test_viewport.py (TestViewportWithPostGIS.test_find_shape_at_point): Adapt to the new shapeids in the landmarks table * test/test_transientdb.py (TestTransientTable.run_iceland_political_tests) (TestTransientTable.test_transient_joined_table): Add tests for the new table methods and new keywords arguments. * test/test_postgis_db.py (TestPostGISTable.test_read_row_as_dict_row_count_mode) (TestPostGISTable.test_read_value_row_count_mode) (TestPostGISTable.test_row_id_to_ordinal) (TestPostGISTable.test_row_oridnal_to_id): New test for the new table methods and the new arguments (TestPostGISShapestorePoint.test_shapes_in_region) (TestPostGISShapestorePoint.test_shape_raw_data) (TestPostGISShapestorePoint.test_shape_points) (TestPostGISShapestorePoint.test_shape_shapeid) (TestPostGISShapestorePoint.test_all_shapes) (TestPostGISTable.test_simple_query) (TestPostGISTable.test_simple_query) (TestPostGISTable.test_simple_query) (TestPostGISTable.test_read_value) (TestPostGISTable.test_read_row_as_dict): Adapt to the new shapeids in the landmarks table * test/test_memory_table.py (TestMemoryTable.test_read_row_as_dict_row_count_mode) (TestMemoryTable.test_read_value_row_count_mode) (TestMemoryTable.test_row_id_to_ordinal) (TestMemoryTable.test_row_oridnal_to_id): New test for the new table methods and the new arguments * test/test_dbf_table.py (TestDBFTable.test_read_row_as_dict_row_count_mode) (TestDBFTable.test_read_value_row_count_mode) (TestDBFTable.test_row_id_to_ordinal) (TestDBFTable.test_row_oridnal_to_id): New test for the new table methods and the new arguments
Remove some unnecessary imports
* Thuban/Model/transientdb.py (TransientTableBase.Width): The type constants in the column objects are the standard ones defined in the table module. * test/test_transientdb.py (TestTransientTable.test_transienttable_to_dbf): New. Test whether exporting transient tables as DBF works. This should catch the bug just fixed in TransientTableBase.Width.
* Resources/XML/thuban-0.9.dtd: New. This will become the DTD for 0.9. * Thuban/Model/transientdb.py (TransientJoinedTable.JoinType): New. Return the join type * Thuban/Model/save.py (SessionSaver.write_session): Use new 0.9 DTD (SessionSaver.write_data_containers): Save the join type for joined tables * Thuban/Model/load.py (SessionLoader.__init__): Add the new 0.9 namespace (SessionLoader.start_jointable): Handle the jointype attribute * test/test_load_0_8.py: New. Effectively a copy of test_load.py as of Thuban 0.8. These are now tests to determine whether Thuban can still read files generated by Thuban 0.8 * test/test_load.py (LoadSessionTest.dtd) (TestSingleLayer.file_contents) (TestLayerVisibility.file_contents, TestLabels.file_contents) (TestLayerProjection.file_contents) (TestRasterLayer.file_contents, TestJoinedTable.file_contents) (TestJoinedTable.file_contents) (TestLoadError.file_contents): Update for new DTD (TestJoinedTable.file_contents, TestJoinedTable.setUp): Add test for new join type attribute * test/test_save.py (SaveSessionTest.dtd) (SaveSessionTest.testEmptySession) (SaveSessionTest.testSingleLayer) (SaveSessionTest.testLayerProjection) (SaveSessionTest.testRasterLayer) (SaveSessionTest.testClassifiedLayer) (SaveSessionTest.test_dbf_table) (SaveSessionTest.test_joined_table): Update for new DTD (SaveSessionTest.test_joined_table): Add test for new join type attribute
* test/test_transientdb.py (TestTransientTable.test_transient_joined_table_same_column_name): Update to reflect the new behavior (TestTransientTable.test_transient_joined_table_with_equal_column_names): Update to reflect the new behavior (TestTransientTable.test_transient_joined_table_name_collisions_dont_modify_in_place): New test case for a bug which modified the column objects in place * Thuban/Model/transientdb.py (TransientJoinedTable.__init__): Update doc-string (TransientJoinedTable.create): Do not modify the column objects of the input tables in place and copy all columns of the input tables into the joined table after all.
(TransientJoinedTable.create): If duplicates in left and right tables column names are found, append '_' (underscores) to the name until it is unique. Create always new internal names for the resulting table and reference columns in the join statement with <table>.<column>
* test/test_transientdb.py (TestTransientTable.test_transient_joined_table_same_column_name): New. Test whether joining on columns with the same names in both tables works. * Thuban/Model/transientdb.py (TransientJoinedTable.create): Make sure to use the right internal names even when joining on field with the same names in both tables. Also, detect duplicate names in the joined table correctly.
Remove the Precision methods again. They're too DBF specific to be part of the table interface and they're only used in table_to_dbf anyway. * Thuban/Model/transientdb.py (TransientTableBase.Width):Use a fixed precision of 12 for doubles. (TransientTableBase.Precision): Removed (AutoTransientTable.Width): Delegate to self.table. * Thuban/Model/table.py (DBFTable.Precision) (MemoryTable.Precision): Removed. (MemoryTable.Width): Use a fixed precision of 12 for doubles. (table_to_dbf): Use a fixed precision of 12 for floats unless the column object specifies something else.
TrannsientTableBase.Width, TransientTableBase.Precision): Return column width and precision.
(TransientTable, TransientJoinedTable, AutoTransientTable): mix-in TitledObject and call its init-method with a default title. Remove Title() method.
(TransientJoinedTable.__init__): Added "outer_join = False" as optional parameter. (TransientJoinedTable.create): If outer join is true, perform a "LEFT OUTER JOIN" instead of "JOIN", which preserves all records of the left table. Records not matching are filled with 0 / None.
(TransientJoinedTable.__doc__): Typo fixed.
Give the tables titles so that the GUI can display more meaningful names. For now the titles are fixed but depend on e.g. filenames or the titles of the joined tables. * Thuban/Model/transientdb.py (TransientTable.Title) (TransientJoinedTable.Title, AutoTransientTable.Title): New. * Thuban/Model/table.py (DBFTable.Title, MemoryTable.Title): New. * test/test_transientdb.py (TestTransientTable.test_auto_transient_table_title): New. Test for the Title method (TestTransientTable.test_transient_joined_table) (TestTransientTable.test_transient_table): Add test for the Title methods * test/test_memory_table.py (TestMemoryTable.test_title): New. Test for the Title method * test/test_dbf_table.py (TestDBFTable.test_title): New. Test for the Title method
Implement a way to discover dependencies between tables and shapestores. * Thuban/Model/transientdb.py (TransientTableBase.Dependencies) (TransientJoinedTable.Dependencies) (AutoTransientTable.SimpleQuery): New. Implement the dependencies interface (TransientJoinedTable.__init__): Keep tack of the original table objects in addition to the corresponding transient tables. * Thuban/Model/table.py (DBFTable.Dependencies) (MemoryTable.Dependencies): New. Implement the dependencies interface * Thuban/Model/data.py (ShapeTable): New. Helper class for ShapefileStore (ShapefileStore.__init__): Use ShapeTable instead of AutoTransientTable (ShapefileStore.Table, ShapefileStore.Shapefile): Add doc-strings (ShapefileStore.FileName, ShapefileStore.FileType): New. Accessor methods for filename and type (ShapefileStore.Dependencies): New. Implement the dependencies interface (DerivedShapeStore): New class to replace SimpleStore. The main difference to SimpleStore is that it depends not on a shapefile but another shapestore which expresses the dependencies a bit better (SimpleStore.__init__): Add deprecation warning. * test/test_dbf_table.py (TestDBFTable.test_dependencies): New. Test for the Dependencies method. * test/test_memory_table.py (TestMemoryTable.test_dependencies): New. Test for the Dependencies method. * test/test_transientdb.py (TestTransientTable.test_auto_transient_table_dependencies): New. Test for the Dependencies method. (TestTransientTable.test_transient_joined_table): Add test for the Dependencies method. * test/test_session.py (TestSessionSimple.setUp) (TestSessionSimple.tearDown): New. Implement a better way to destroy the sessions. (TestSessionSimple.test_initial_state) (TestSessionSimple.test_add_table): Bind session to self.session so that it's destroyed by tearDown (TestSessionSimple.test_open_shapefile): New. Test for OpenShapefile and the object it returns
(AutoTransientTable.SimpleQuery):
The right argument may not be a string, it could also be a Column.
* Thuban/Model/transientdb.py (TransientTableBase.ReadRowAsDict): Add comments about the optimizations used. (AutoTransientTable.ReadValue, TransientTableBase.ReadValue): New. Implement the ReadValue table interface method. * test/test_transientdb.py (TestTransientTable.run_iceland_political_tests) (TestTransientTable.test_transient_joined_table): Add tests for ReadValue
(AutoTransientTable.SimpleQuery): New. Simply delegate to the transient table's version.
(TransientTableBase.SimpleQuery): Implement a simple query method for the query dialog (TransientTableBase.create): Add an INTEGER PRIMARY KEY that holds the row index or shapeid. (TransientTable.create): Insert the right value of the row index (TransientJoinedTable.create): Copy the row index of the left table to the joined result table
Convert all table users to use the new table interface. This only covers Thuban itself, not GREAT-ER or other applications built on Thuban yet, so the compatibility interface stays in place for the time being but it now issues DeprecationWarnings. Finally, the new Table interface has a new method, HasColumn. * Thuban/Model/table.py (OldTableInterfaceMixin): All methods issue deprecation warnings when they're. The warnings refer to the caller of the method. (OldTableInterfaceMixin.__deprecation_warning): New. Helper method for the deprecation warnings * test/test_table.py: Ignore the deprecation warnings for the old table in the tests in this module. The purpose of the tests is to test the old interface, after all. * test/test_transientdb.py (TestTransientTable.run_iceland_political_tests): Use the constants for the types. Add a test for HasColumn (TestTransientTable.test_transient_joined_table): Adapt to new table interface. Add a test for HasColumn (TestTransientTable.test_transient_table_read_twice): Adapt to new table interface * Thuban/Model/transientdb.py (TransientTableBase.HasColumn) (AutoTransientTable.HasColumn): Implement the new table interface method (AutoTransientTable.ReadRowAsDict, AutoTransientTable.ValueRange) (AutoTransientTable.UniqueValues): Adapt to new table interface * Thuban/Model/layer.py (Layer.SetShapeStore, Layer.GetFieldType): Adapt to new table interface * test/test_layer.py (TestLayer.open_shapefile): Helper method to simplify opening shapefiles a bit easier. (TestLayer.test_arc_layer, TestLayer.test_polygon_layer) (TestLayer.test_point_layer): Use the new helper method (TestLayer.test_get_field_type): New. Test for the GetFieldType method * test/test_dbf_table.py (TestDBFTable.test_has_column): Test for the new table method * test/test_memory_table.py (TestMemoryTable.test_has_column): Test for the new table method HasColumn
Convert the table implementations to a new table interface. All tables use a common mixin class to provide backwards compatibility until all table users have been updated. * Thuban/Model/table.py (OldTableInterfaceMixin): Mixin class to provide backwards compatibility for table classes implementing the new interface (DBFTable, MemoryTable): Implement the new table interface. Use OldTableInterfaceMixin as base for compatibility (DBFColumn, MemoryColumn): New. Column description for DBFTable and MemoryTable resp. * test/test_dbf_table.py: New. Test cases for the DBFTable with the new table interface. * test/test_memory_table.py: New. Test cases for the MemoryTable with the new table interface. * test/test_table.py: Document the all tests in this file as only for backwards compatibility. The equivalent tests for the new interface are in test_memory_table.py and test_dbf_table.py (MemoryTableTest.test_read): field_info should be returning tuples with four items (MemoryTableTest.test_write): Make doc-string a more precise. * Thuban/Model/transientdb.py (TransientTableBase): Convert to new table interface. Derive from from OldTableInterfaceMixin for compatibility. (TransientTableBase.create): New intance variable column_map to map from names and indices to column objects (TransientTable.create): Use the new table interface of the input table (AutoTransientTable): Convert to new table interface. Derive from from OldTableInterfaceMixin for compatibility. (AutoTransientTable.write_record): Removed. It's not implemented yet and we still have to decide how to handle writing with the new table and data framework. * test/test_transientdb.py (TestTransientTable.run_iceland_political_tests) (TestTransientTable.test_transient_joined_table): Use the new table interface
* Thuban/Model/transientdb.py (TransientTableBase.__init__): New instance variable read_record_last_result (TransientTableBase.read_record): Make sure reading the same record twice works. The implementation uses the new instance variable read_record_last_result * test/test_transientdb.py (TestTransientTable.test_transient_table_read_twice): New test case for the above bug-fix.
(TransientDatabase.close): Set self.conn to None after closing the connection to make sure it's not closed twice
Next step of table implementation. Introduce a transient database using SQLite that some of the data is copied to on demand. This allows us to do joins and other operations that require an index for good performance with reasonable efficiency. Thuban now needs SQLite 2.8.0 and pysqlite 0.4.1. Older versions may work but I haven't tested that. * Thuban/Model/transientdb.py: New. Transient database implementation. * test/test_transientdb.py: New. Tests for the transient DB classes. * Thuban/Model/session.py (AutoRemoveFile, AutoRemoveDir): New classes to help automatically remove temporary files and directories. (Session.__init__): New instance variables temp_dir for the temporary directory and transient_db for the SQLite database (Session.temp_directory): New. Create a temporary directory if not yet done and return its name. Use AutoRemoveDir to have it automatically deleted (Session.TransientDB): Instantiate the transient database if not done yet and return it. * Thuban/Model/data.py (ShapefileStore.__init__): Use an AutoTransientTable so that data is copied to the transient DB on demand. (SimpleStore): New class that simply combines a table and a shapefile * Thuban/Model/table.py (Table, DBFTable): Rename Table into DBFTable and update its doc-string to reflect the fact that this is only the table interface to a DBF file. Table is now an alias for DBFTable for temporary backwards compatibility. * Thuban/UI/application.py (ThubanApplication.OnExit): Make sure the last reference to the session goes away so that the temporary files are removed properly. * test/test_load.py (LoadSessionTest.tearDown): Remove the reference to the session to make sure the temporary files are removed.
This form allows you to request diffs between any two revisions of this file. For each of the two "sides" of the diff, enter a numeric revision.
| [email protected] | ViewVC Help |
| Powered by ViewVC 1.1.26 |