SvnToCCase Project on sarovar.org

Overview

SvnToCCase (Unix Name: svn2cc) is a package to convert Subversion Dumps to IBM/Rational ClearCase commands in Batch files plus Zip-Files containing the modifications. It consists of two Java packages: The Manpage of svn2cc.jar follows:
For current documentation and releases, please check SvnToCCase Project on sarovar.org.

SvnToCCase

NAME
SYNOPSIS
NOTICE
DESCRIPTION
BELLS AND WHISTLES
OPTIONS
FILES
SEE ALSO
TODO
AUTHOR

NAME

SvnToCCase − Convert Subversion dumps to ClearCase commands

SYNOPSIS

java [java-vm-options] −jar svn2cc.jar subversion-dumpfile [−vq] [−t subversion-trunkname] [−b clearcase-branchname] [−c ‘‘after-checkin-cmd’’] [−d path-to-viewroot] [−s[d|f] checkout-selector (global/file/dir)] [−fo ‘‘revision-comment-format’’] [−x exclude-element]* [−i include-element]* [−coa extra-checkout-args]* [−cia extra-checkin-args]*

Spaces are mandatory between options and arguments.

NOTICE

Licensed under the BSD License
(http://www.opensource.org/licenses/bsd-license.php)
by Stefan Hegny, hydrografix Consulting GmbH,
Frankfurt/Main, Germany
and others, see http://svn2cc.sarovar.org
November 2005-

If you are interested in using the svndump classes by programming instead of using everything to create ClearCase scripts, see the svndump source files.

The original jar file contains the full eclipse project. It can be used as a source for creating a project in your workspace, or you can include it as library into a project, using as source attachment the same jar.

ClearCase is a registered product name of IBM/Rational.

DESCRIPTION

Extract information from Subversion dump, filter, generate Windows-.bat files invoking ClearCase commands to replay the actions from the Subversion dump, including possibly merging.

If using the playall.bat, it requires a ‘‘unzip’’ program understanding the −o (for overwriting existing files) and −d parameters with forward slashes ‘‘/’’ as path separators (for using the −d parameter to SvnToCCase). E.g. the cygwin unzip. You can also use any other zip decompressor if you manually call the checkout-/checkin-scripts.

The input dump file usually is generated with a command like

svnadmin dump --incremental -r from-revision:to-revision repository > subversion-dumpfile

If you do not specify the --incremental to the dump, the output will try to create all elements present in the dump as ClearCase elements. Otherwise only modifications will be noted (which I assume is the normal thing).

It is assumed that the ClearCase view root is the base of the trunk or branches in the Subversion repository. The output is assumed to be run at the ClearCase view root, but this can be changed using the -d parameter.

Scenario Description

Assume a project is developed using ClearCase at a site A. A team does development of parts of the project at site B using Subversion. Every while, team B brings its modified files in to A and checks-in the changes. They also pull the files modified at A back into the Subversion repository.

The comments on check-in may range from ‘‘one week worth of changes’’ to a list of actual changes. Hovever all changes at a file will be one single check-in.

So why not play back the individual changes to a file into ClearCase? This is a great idea especially if no one at A will modify those files usually modified at B. Everybody at A can then view the individual changes plus their comments in ClearCase.

For the ‘‘playing back’’, the from-revision for the dump of course is one after the commit of the last synchronization from A, the to-revision HEAD.

Now, what happens if someone at A has modified a file that also was modified at B? Obviously, the check-in commands will fail with the message ‘‘the checked-out version is not the lastest on the branch’’. This can be seen by the scripts stopping on the cleartool error code. In that case, and there is work left to be done, the file merges.zip contains merge commands in the file mergers.bat from which you can select the appropriate one. It also contains check-in commands using either the comments_revision.txt from the batches.zip, if only one commit exists for the file, or a specific comments file containing an aggregation of all commit comments for all revisions that were now merged into the new ClearCase version. As the new ClearCase version is probably not identical to the last Subversion revision, the after-checkin-commands are not run after the checkin, so that you know you have to pull these changes over to B.

It is safe to run the mergers.bat every time after you run playall.bat, however, this may yield lots of errors for those files that already were processed successfully because no check-out is done here.

In case any of the files being synchronized is currently checked-out reserved at A, you have to wait anyway until it is checked-in, or think of a different solution.

For that synchronization stuff, here are two bash scripts pulling a list of the files not having the ‘‘moving label’’ and applying the label to them: (use on view root with . mklist.sh > new_list).

It is probably a good idea to have a snapshot view that is exclusively used for the synchronization, i.e. do not edit elements here and do not create any view-private files here. This makes it easier to pull a complete tree over from time to time without having a lot of garbage in there.

mklist.sh:
cleartool find * -type f -vers ’!lbtype(MY_MOVING_LABEL) && version(/
branchname/LATEST)’ -exec ’bash -e mklist_aux.sh’

mklist_aux.sh:
# turn around backslashes
aktn=‘echo $CLEARCASE_PN | sed -e ’sM\\\\M/Mg’‘
# only include non-empty files
if [ -s ’’${aktn}’’ ]
then

# apply label
cleartool mklabel -replace MY_MOVING_LABEL ’’${CLEARCASE_PN}’’ > /dev/null
# print, omitting lost+found
echo ${aktn} | grep -v lost\+found

fi

When you create a zip archive containing the listed files, unzip that into a Subversion checkout, possibly add files to Subversion that are new, and then commit, then you should have done the synchronization ‘‘the other way’’ from ClearCase to Subversion (however, losing the ClearCase history, that is another issue). The dump for the next run of SvnToCCase then starts one revision past that commit.

BELLS AND WHISTLES

The concepts of ClearCase and Subversion are quite different, so the few things that can safely be done and ported into ClearCase are:

- modifying files
- adding new files or directories
- removing files or directories
- moving/renaming files or directories (this already is a kind of guessing if a copy from the same branch plus a delete of the copy source is within one revision).
A file that is moved may or may not at the same time be modified. This difference is now honored.

Especially noteworthy seem svn copy operations. You can of course use the branching/labeling/merging in Subversion, however here is currently no way to synchronize that over into ClearCase.

The view into which the playback is run should be set up so that checked-in elements can be seen (i.e. branch/LATEST should be seen). If this is not the case, subsequent check-outs may all check-out the same version and not the one from the last check-in, which in turn will make the check-in fail because ‘‘the checked-out version is not the lastest on the branch’’.

Even if you use the label mechanism with −s/−c, this still requires the label to be applied to the correct version after checkin. This could be overridden by specifying a branch/version in the after-checkin-command, this is left an exercise to the readers...

OPTIONS

−q, −v

Be more quiet or more verbose

−t subversion-trunkname

Filter only elements in the subversion-trunkname (or complete branchname), stripping that trunk- or branch-name from the head of the element name. Specifying an empty trunk- or branch-name (i.e., not stripping anything) is currently not supported.
All comparisons are case-sensitive.

Default value: trunk

Sample usage: −t branch/dev_for_sthing to include only stuff from branch dev_for_sthing.

−b clearcase-branchname

In merge script, use this branch’s LATEST element as the contributor to the merge.
This is also where we expect all things to be after check-in.

Default value: main

Sample usage: −b /main/For_Sthing_xx to use the latest version from that branch for merging.

−c ‘‘after-checkin-cmd’’

Command to be executed after a successful checkin (but not for checkins in the merge script). This will usually consist of more words and thus be quoted. In the command, {} will be replaced with the double-quoted element name (which will contain forward slashes ‘‘/’’ as path separator).

Default value: no command executed.

Sample Usage: −c ’’cleartool mklabel -replace MY_MOVING_LABEL {}’’ to move a label indicating this file is in-sync.

−d path-to-viewroot

If not running at view root level, prepend this path to all element names passed to cleartool in the scripts and make the unzip unpack the contents of a revision to that directory.

Default value: run at view root.

Sample Usage: −d ../.. if the result will be run from a sub-sub-directory of the view root (maybe a view-private directory within a VOB).

−s checkout-selector

−sf checkout-selector-files

−sd checkout-selector-dir

When checking out, use this version from ClearCase, possibly overriding the one currently in the view. The parameter is appended to the element name on the checkout command separated by @@. You could specify a label attached to the version you want to checkout.

If you specify −s, the specified selector is set for both file and directory elements. −sf only will apply to file elements, while −sd only will apply to directories. The latest on the command line will be the strongest.

Default value: checkout version currently selected in the view.

Sample Usage: −sf /MY_MOVING_LABEL to run the scripts on a dynamic view, but always continue at the last synchronization point (set with the −c arg sample above) on file elements.

−fo ‘‘revision-comment-format’’

Format of the line added to the subversion revision comments for the clearcase checkin comments.

May contain
− verbatim text
%A% subversion revision author
%N% subversion revision number
%Z% subversion revision datestamp (GMT/UTC/Zulu)

Default value: [svn:%N%:%A%,%Z%]

Sample Usage: −fo ‘‘’’ to not add any subversion specific stuff.

−x exclude-element

−i include-element

Specify which commits of which files are included. The names are already stripped of the trunk- or branch-name specified by -t.
If only −i arguments are present, only the specified elements are included.
Otherwise, all elements matching a −x are excluded, except if a −i will include them (no matter in which sequence the arguments are listed).

The syntax for exclusion and inclusion:

element-name[*][<|>revision-number]

Note the wildcard asterisk is only permitted at the end of the name. Only revisions below or above a given number can be selected using the trailing < or >.
All comparisons are case-sensitive.

Default value: nothing excluded.

Sample Usage:
−x VOB_1/* −i VOB_1/dir2/*
to exclude VOB_1, except VOB_1/dir2 and contents which is included too.

−coa extra-checkout-args

−cia extra-checkin-args

Additional options for the ClearCase check-out/-in. Do not specify options related to comments or versions. A useful checkout-option may be -unr so that the checkout succeeds even if someone else has checked-out the file reserved. Useful checkin-options are -pti (preserve modification time) or -ide to check-in identical (i.e. non-modified) files.
You can either specify multiple −c[io]a arguments and/or specify multipe args within a quoted string.

Default value: no extra args.

Sample Usage:
−coa -unr
to make the check-out succeed even if currently someone holds the file reserved.
−cia -pti
to preserve the timestamp

FILES

revision.zip

All files modified for revision number revision plus a directory sav containing, for completeness reasons, the check-out, check-in and comments files also contained in batches.zip.

The modification times of the files in the zip archive is set to the local time corresponding to the UTC timstamp for the commit in the dump.

batches.zip

Itself contains several files.

comments_revision.txt

Commit-Comments for revision number revision.
These are the comments for the Subversion revision plus a line
[svn:
revision-number:user,date-time]
which can be changed using the −fo1 parameter. Date-time is the (UTC/GMT/Zulu) timestamp from the dump.

chout_revision.bat

Batch-file performing check-out of all elements for revision number revision. Will stop for user input if cleartool returns non-zero, and then continue.

chin_revision.bat

Batch-file performing check-in of all elements for revision number revision. Will stop for user input if cleartool returns non-zero for error, and will call the specified after-checkin-command for the element if it returns zero for ok.

playall.bat

Sequentially call check-out, unzip and check-in for all revisions using the chout_(chin_,comments_)revision.bat(.txt)

revref.csv

Overview of elements and revisions in which something was done to them.

merges.zip

Itself contains several files.

ci_seqno_filename.txt

Aggregated comments for file filename. Only for files having more than one commit. If only one commit is present, the comments file for that commit is used.

mergers.bat

Batch Script performing all possible merges. Possibly a bit much for unconditionally executing it. May be useful to copy lines out of (merge and checkin) for those files that have to be merged − only use after the playall.bat has completed!

SEE ALSO

http://subversion.tigris.org
http://www.ibm.com/software/awdtools/clearcase/index.html
http://sarovar.org/projects/svn2cc
http://www.cygwin.com

TODO

Alternative output:
- Make sh script instead of bat for Un*x or cygwin environments.
- Call cleartool directly from the dump. This would make merging easier because we could detect on which elements it is necessary.

Use JavaHL instead of a dump file.

Perform the after-checkin-command only after the last of the revisions for a file was successfully checked in. Otherwise intermediate steps are labelled which are not identical to the final one. However, this imposes the problem that using the −s checkout-selector, when used with the same label that is applied in the after-checkin-command, will always try to checkout the originally labelled version (which is no longer the ‘‘latest on its branch’’ after the first check-in) and thus, the checkout-selector would only have to be used on the first check-out.

AUTHOR

Stefan Hegny
hydrografix Consulting GmbH
Röderichstr. 8
D-60489 Frankfurt a.M.
shyffm@users.sarovar.org
http://sarovar.org/projects/svn2cc/