Par2: Synchronization and DCT "direction" 1998-11-26 -- Alexey Solovjev The issue of direction
=======================
Yes, this moment is quite difficult to understanding for first time: the OLD
dictionary should be specified as TARGET and NEW one as SOURCE. Opposite to
expectations. From my view point, this could be explained with following
argument. Really, the synchronizer works with THREE(!) dictionaries:
1) INITIAL dictionary that defines current database structure;
2) DESTINATION dictionary that defines new database structure; and,
3) RESULT dictionary that defines database structure after synchronization.
When you specifying settings for the Synchronizer DESTINATION dictionary
is a SOURCE and RESULT dictionary is a TARGET. At the beginning of the
synchronization the RESULT dictionary is equal to the INITIAL one. While
you resolving differences in the Synchronizer window, the state of the
RESULT dictionary is changing. In ideal case, as a final point the RESULT
and DESTINATION dictionaries should be equal (i.e. the RESULT dictionary
has gone the way from INITIAL to DESTINATION) but this is not obligate:
- not all tables could be selected for synchronization;
- some differences could be ignored.
Therefore at final point when you presses the Ok button the RESULT dictionary
reality of 3 dictionaries is obvious. The RESULT dictionary makes work of the
Synchronizer more logical. If you've added new field, in the Synchronizer
window you ADDs it to the RESULT dictionary. If something has been changed,
the old value in the RESULT dictionary is replacing by new one. Etc.
The Convert Program
===================
Yes the conversion program is not an ABC program. There are two reasons for
that. (1) It should be as independent from all other staff as possible. (2)
Most methods required for conversion purposes have no analogs in FileManager
class of ABC. Therefore derivation of CVT classes from ABC classes would be
absolutely nominal.
Designing the conversion program's core I've had following goals:
1) The conversion program is for ISAM file drivers primarily but it's need
to allow work with SQL tables too with minimal changes in sources, i.e.
with overriding minimal methods in derived classes.
2) The conversion program should be as flexible and efficient as possible.
For example, if the only modification in the DBF table is a new key, the
program should build that new key only without rebuilding of the table
itself.
3) The conversion program should do all possible changes automatically. E.g.,
if a new key has been added with a new field as auto-increment component
(last component), the program should build the correct values for that
component.
To achieve these goals the conversion program is based on several base
classes with numerous virtual methods that can be overridden either by the
program generator or by manually by the programmer.
The conversion process is partitioned to a set of elementary _tasks_ which
are executing consecutively. Every task is performing by its own virtual
method that can call other virtual methods.
A set of tasks for specific table being converted is a value of the Task
field of the Convert class (or its derivations). This value is a bitmap.
Bits equal to 1 correspond to tasks which should be executed. The main
loop of the conversion process searches for next bit in the Task field
set to 1 and performs corresponding elementary task. This way allows to
skip actions unnecessary for specific tables. There are 3 default sets of
tasks declared in the C5CVT___.INC file:
- TASK:DefaultSimple : for conversion of ASCII, DOS or BASIC files;
- TASK:DefaultISAM : for conversion of ISAM files;
- TASK:DefaultSQL : for conversion of SQL tables.
It's need to say that the conversion program generator not provides full
support for SQL because preferred and suggested way to modify SQL tables is
to use DBA tools or scripts generated by the Synchronizer. Therefore the
programmer needs to declare required virtual methods by hands. Say, to
build new indexes following method could be declared:
Cvt:Tratata.BuildKey PROCEDURE (KEY K)
CODE
SELF.File {PROP:SQL} = 'CREATE UNIQUE INDEX ' & ...
IF ERRORCODE() = 0
RC = RC:Ok
ELSE
SELF.ErrorMessage ('Unsuccessful BUILD for ' & K {PROP:Label})
RC = RC:Error
END
RETURN RC
The most simple way to change SQL table's structure is to declare AlterTable
method, say:
Cvt:Tratata.AlterTable PROCEDURE ()
CODE
SELF.File {PROP:SQL} = 'ALTER TABLE ' & SELF.File {PROP:Name} & ' ...'
...
> HA! It works. Well, nearly. There's a field name I changed, and somehow
> that didn't get assigned. But this program is object oriented! For each
> table, there's a Cvt:Tablename.Assign method which starts out with:
>
> CLEAR( newtable.Record)
> newtable.Record :=: oldtable.Record
>
> and is followed by a bunch of commented out assignment statements. I'll
> just remove the "!" from this one here, and assign it to the old field...
> By golly, the silly thing now works!
This means that new name of renamed field has not been matched with old
name. The program generator should produce correct line in the Assign
method.
Backup
======
Very difficult thing is backup. The problem occurs if a file is converting
on the place especially for TPS files. TPS files have multi-table structure
and there is no legal (i.e., available for end user) way to obtain tables
names without opening one of them. This makes practically impossible to use
the "replace" schema of converting TPS files on the place or it is too
inefficient.
Therefore backup schema has been chosen. Backup is optional if the source
and result tables have different physical names. If the table is converting
on the place it should be backed up. The backup directory MUST be other than
directory(-ies) where source and/or result tables are located.
Conversion speed
================
The conversion of multiple files could take long time. Therefore the
program has been designed to allow concurrent processes to run. Opposite
side of that is some speed penalty. There are two ways to increase the
speed of conversion:
1) The field RecordsPerQuant of Convert class can be set to larger value in
the Init method. Default value is 100.
2) The conversion of every table is a simple final state machine. There are
5 states only:
- STATE:Initial - obvious, the initial state;
- STATE:Completed - obvious, the automate takes this state after finishing
of conversion;
- STATE:Continue - this state means that the long task (usually, rebuilding
of the table) must be continued after re-dispatching;
- STATE:NextTask - this state means that the conversion should begin to
find and execute next task after re-dispatching;
- STATE:NoIdle - this state means that the conversion should begin to
find and execute next task WITHOUT re-dispatching.
By default, after finishing of every task the field State is setting to
STATE:NextTask. This allows to re-dispatch the conversion program and
gives a chance to other processes to run. Changing of the State field to
STATE:NoIdle skips re-dispatching and speeds up the conversion execution.
Printed November 21, 2024, 7:01 am This article has been viewed/printed 35202 times. |