Help with ClearCase to Teamforge Subversion migration!

Jagadesan jayabalan's picture

We are plannning to migrate our source code from UCM Clearcase to Teamforge subversion.

We need to check,analysis some tool cc2svn,poloriod tool etc.

Is any one already done the migration, can you share your experience and ways to migrate

Thanks.

17 Answers

Prabhu's picture
Prabhu replied on November 21, 2010 - 9:10pm.

Hai Jack

I am also planning for the same and need information on this,

1. Why svn is a Repository level versioning tool na d what is the advantage of that ?

2. What about the Tagging in svn .... if i want to label a few files elements .. does it track the entire directory structure while labeling ?

Thanks

Hayley's picture
Hayley replied on January 24, 2011 - 10:38am.

Hello both,

We are also looking to do the same!

I have a few questions for you:

What size development teams do you guys have?

How many projects do you have on the go at once?

Do you have any large development projects that require interproject deliveries as they all modified shared/common code?[/ul]

What sort of analysis/investigation did you do for migrating?

And what were your reasons for moving?

How far have you got so far?

Thanks,

Hayley

Marc Girod's picture
Marc Girod replied on January 24, 2011 - 3:48pm.

Hayley wrote: "We are also looking to do the same!"

I would recommend that people might consider the moves:

1. from UCM to ClearCase: ClearCase has a lot of unexploited resources, once one gets back from UCM

2. from UCM to anything else but subversion!

Subversion is a pretty radical denial of SCM, with its throwing away of configuration items (for directory trees)

Marc

Bob Aiello's picture
Bob Aiello replied on January 24, 2011 - 7:33pm.

Hi,

Have you guys thought about the Use Case that you will employ for branching? This Subversion Branching is rather different than ClearCase (even more so than UCM).

Bob

jrep's picture
jrep replied on January 24, 2011 - 7:55pm.

The primary reason Subversion branches and tags are the way they are is so that they can be dirt cheap: you can branch or tag any amount of data, any directory structure, and configuration in a single operation, in O(1) time (just a few seconds).

If you want to mark only selected file/directory elements, you can use a "property". But there's currently no automated way to then use those properties for functions like checkouts. In general, Subversion presently lacks the power functions of ClearCase config-specs, or the "cleartool find" ability to search the history. The Subversion focus is primarily on "providing what you need, fast and reliably."

jrep's picture
jrep replied on January 24, 2011 - 8:04pm.

The Subversion "reintegrate" branch merge strategy is fairly similar to the UCM model: create a branch for team, feature, or developer; work on that branch, occasionally merge out from trunk to stay current; eventually merge back down to trunk to deliver your work to the release stream.

For more details, check the Subversion book, http://bit.ly/hQgbCW] Keeping a Branch in Sync section.

Bob Aiello's picture
Bob Aiello replied on January 24, 2011 - 8:09pm.

hey Jack - thanks for replying! Could we elaborate a little more please?

ClearCase (both base and UCM) have some pretty impressive branching capabilities. I am trying to get my arms around how to provide the same functionality in Subversion. Can we discuss this a bit further? I would also be delighted to work with you on an article or video cast to discuss this further.

Folks - in case you didn't notice we have the Jack Repenning, CTO of Collabnet, helping us out in this discussion.

jrep's picture
jrep replied on January 24, 2011 - 9:16pm.

I used ClearCase for ten years or more, even worked at Rational for a while. I'm very aware of the power of the advanced features, and I'm not at all claiming that Subversion is point-for-point equivalent.

But at the same time, I'm also aware of a lot of limitations of ClearCase. It's very expensive in terms of compute resources and administrative support, for example. Some of the coolest features end up not being used at all, in many shops, because of platform compatibility issues (compatibility, incompatibility, and inter-compatibility), or problems when people take their laptops out of the network, or evolutions in language-processor design that violate the config-rec model. High-power items like config specs can be a lot of rope, and we've all hung ourselves more than once. Delightfully visual representations like the version tree can knacker the server(s) in no time.

The Subversion community began with a notion of how to do branches and tags really cheaply. In as much as I once worked in a large, globally distributed ClearCase shop where we had two engineers fully employed, 24x7, just making release tags, "cheap tags" gets my attention! With Subversion's cheap tags, you can safely tag what goes *into* a build, and let the build and test cycles tell you whether it's any good. And if it's not, you can safely remove the branch or tag, or file it away in some subdirectory where it needn't distract people--whatever fits your process--and none of these tag creations and adjustments need take longer than five seconds. Ever.

An additional advantage that should interest this audience is that the "global revision number" (which is the heart of the cheap-tag secret) is also a "global change number," suitable for tracking purposes in a sophisticated CM system. The appropriate Subversion commands can take either a "--revision REVISION" flag (which means "the that-version of everything") or a "--change REVISION" (which means "just the changes that went into that one commit"). In this way, the revision/change number can be tracked, reviewed, applied to related branches, and all the other change-centric stuff that can make a real difference for the CM process.

It's certainly true that ClearCase, as the latest product of over 30 years' evolution, has a head start over Subversion in some areas. CollabNet has attempted to help out, by supplementing the community's core contributions, with such add-ons as branch-and-merge wizards, change-based tracking tools, and the like, but there are still some holes and rough spots. But some of the simplicities are intended: just because you want something, such as change-tracking, does not necessarily mean you want to spend an arm and a leg to get it. A little rethinking about how you really want to use changes, a novel tool like cheap tags, and you may have a better solution than you did before. Different, yes. But better.

Hayley's picture
Hayley replied on January 25, 2011 - 10:37am.

Hey guys,

Thanks for your replies.

We are currently using UCM, which to be honest suits us fine, but the business would like us to move across onto Subversion to save costs and increase productivity.

How does Subversion cater for large development projects where you have multiple developers changing the same file at the same time and frequently perform deliveries and rebases (more than twice daily)? We also often deliver between UCM projects so the changes can be reflected in one of the later releases of the project - can it cope with this?

And does Subversion have the facility to lock down the repository so we can have control over which people are allowed to 'join projects', make changes to elements, create labels and branches etc?

Also, how does Subversion cope across the network? With ClearCase, some of our offside teams using dynamic view experience performance issues due to the network latency. Does the same happen with Subversion? Or does it work in a similar fashion to ClearCase 'snapshot views' and is not so network dependant?

Thanks,

Hayley

pdhaggerty's picture
pdhaggerty replied on January 25, 2011 - 10:50am.

Too bad on going to SVN from CC. CC works just fine with the Remote Client for remote users, just like subversion. I see going to SVN as a backwards step and will require the installation of many other tools to maintain the working ENV that you had. plus you will need to add a UCMSVN tool (Exists on Collabnet) to get anthing approaching the fantastic branch management that UCM provides. I know this since I created a large scale ENV still inuse at a major insurance company (1000+ vobs Using a top level PVOB with child PVOBS then VOBs to handle the inheritence. contact me if you need help with the CC to SVN, I've done that also but not with the tool, I wrote my own, just CC scripts then imports to SVN, very straight forward.

dani_cv's picture
dani_cv replied on January 25, 2011 - 11:51am.

Hi Jack,

my few cents here...

Yes, I have used polarion cc2svn tool.

We were using ClearCase UCM. Business decided to move to SVN for a uniform SCM solution across all businesses.
We used cc2svn tool to convert the CC repo to SVN.

Here is what we got after conversion. We had thousands of development and sub branches branches in UCM which has taken atleast 5 days to move over (dump and load)
- It has created more than 100,000 (global svn) revisions
- When I tried to open a revision tree, it never came up (offcourse, SVN takes long time to load the tree for the first time, may be I could have waited)
- It has created tags for each dev and sub branches - 1000s of them in repository after the conversion
Finally we have decided not to use the converted repo, rather starting a clean SVN repo and imported the last CC source, which is working fine now.
For any old history we can either go to converted SVN repo or ClearCase . But no one uses the cc2svn converted repo now since its not clean, rather they goto old CC (711).

There are good and bad for both. To me, if I need to summarise this is what I would say.
- Subversion is light, easily managed, perfect for small and medium size projects which does not have complex branching support.
- ClearCase is heavy, expensive and complex to manage; But for sure it stands out for its capability to manage complex model. UCM model stands out for any kind of dev/qa/release process. It doesn't matter how big the vob size is, how complex your branching is, CC should be able to handle it.

- I like SVN since it is light weight and most of the operations are really fast as well as it supports file, http, https calls etc.
- If I have complex model (like very frequent releases, patches, merges etc. which needs lot of SCM effort), ClearCase is the solution for me.

Thanks,
Dani.

rjenkins's picture
rjenkins replied on January 25, 2011 - 12:53pm.

Having been at the ground floor of both UCM (I was on the field advisory team working with Brian White on its design) and Subversion I can talk to them both. UCM was designed to be an automated process that would abstract the complexities of ClearCase from the average user. Originally, we hoped to make it more customizable, but that never came to fruition and so you have a very set process. When you automate you basically have to make assumptions as there is a lot that the user knows that automation can't know. For UCM, that meant that anything anyone was assigned to work on would require a stream (only slightly more than just a branch) whether that work really needed isolation or not. Obviously the user or their lead would have been able to make a decision on whether that made sense or not based on knowing what they would be doing.

Because of that automation, it is very important to support not only the idea of branching for isolation, but also the ability to merge the work on that branch with that "published" (i.e., delivered in UCM terminology) by others (and therefore on the integration stream in UCM) as well as be able to merge the "published" work to the "development" branch (i.e., rebased in UCM lingo). The automation drove the need to do these things frequently since all development was by definition isolated.

That model, while simplifying ClearCase, is actually quite complex and constantly hitting the operation most developers want to avoid, merging (or more accurately they want to avoid making the decisions on conflicts). So when you look at development outside of that automated implementation, this model is rarely appropriate for development. Rather a model that isolates work when there is risk rather than just when there is work is more logical and lessens the complexity. It really goes back to a logical best practice that says when you are changing tools, go back to the core requirements and build a process that best utilizes your new tool. Taking a process and trying to make it work in a new tool isn't a good idea. That isn't because it may not have been a good process with the legacy tool, but it was designed to take advantage of the best of that tool and avoid the worst. That's very true of UCM and ClearCase.

Now can you accomplish UCM with Subversion? Yes, but obviously Subversion doesn't have the automation to put it into place that ClearCase does. Subversion is designed to allow the organization to determine their process and makes no assumptions about an out-of-the-box process. Keep in mind that part of why Rational did that with UCM was in reaction to the user pushback on complexity which isn't an issue with Subversion.

In the spirit of being open and direct, it is important to note that Subversion has difficulty with repeated deliveries from the same branch. Subversion's initial target was to replace CVS as the de facto open source, centralized version control tool and CVS had no merge tracking. ClearCase was designed to be an improvement on the work that the development had done in developing DSEE which had merge tracking. Subversion added the first phase of merge tracking in 1.5 and that's no small feature to add so there were going to be limits and this is the main one. Subversion provides great functionality (reintegrate) to handle the merge of a child branch back to the parent when the parent contributed to the child, but the algorithm can't handle the situation where it has been done previously. The work-arounds are to either delete and recreate the child branch to continue on to the next "delivery" point; or you can immediately do a manual merge back to the child branch (assuming no other changes were made on the parent branch). This is something that is targeted to be addressed in one of the next couple of releases.

Subversion has a very powerful and simplified branching model that doesn't have the scalability issues that legacy tools have. It allows for you to quickly and easily isolate work while tracking what you've merged and haven't merged. Organizations have successfully and happily moved to it from any version control tool you can name.

Hope this helps,

Bob

rjenkins's picture
rjenkins replied on January 25, 2011 - 1:08pm.

Forgot to mention that I will be doing a Webcast on the topic of "Subversion Best Practices: Branching Successfully" on February 24th. You should be able to register for it before too long via the CollabNet website. It won't talk about UCM directly, but about branching strategies and Subversion.

tekkie's picture
tekkie replied on January 25, 2011 - 1:53pm.

I have recently decided for the group that I work in, not to migrate from CC to SVN.

The whole hierarchal branch model limitation of SVN is a showstopper. Not too mention other limitations like no partial branches etc.

If you come CC think twice before going down the road of SVN.

Just my $ .02

rjenkins's picture
rjenkins replied on January 25, 2011 - 2:11pm.

It is a great world where we have options where we can find what best suits our needs. Each tool determines what model they want to provide as the baseline for its use, but that doesn't mean you can't do something else rather it may not be mainstream or as easy. UCM is very different than how most people used ClearCase and use ClearCase base for example.

With Subversion, you can definitely do what I would think is meant by partial branches as you can choose whatever point you want to branch and you can prune that branch as you want. That isn't the common or best practice approach to using Subversion, but that comes from the chosen design for Subversion. Subversion's approach partially comes from the fact that you don't have a partial production environment, but rather a complete environment. To help support that, branching is expected to include even things that may not change for this development effort so it is easy to identify what is in production versus trying to tie together a bunch of different branches/tags that may overlap to represent what's in production.

That said, if you want to work on your project as components, then you should branch by components and you'd need to bring the right revisions of the components together to define the full environment. Varying the level that you branch is dangerous in any tool as it is up to you to keep track of what you have on what branch (and that's more difficult for those that follow in trying to understand history). That said, Subversion can handle that, but best practices say that's not something users handle well.

I guess that's where the "hierarchial" point comes into play. That's really just an implementation detail since branches are paths in Subversion. They are meta data in ClearCase so there is no relationship defined nor implied, but I dare say that one is usually understood.

Anyway, there are multiple tools because there are different needs and Subversion isn't the answer to everyone's needs. It can do a lot of what is done with other tools though even if it isn't the standard approach so there is a difference between saying "can't" and "common".

jrep's picture
jrep replied on January 25, 2011 - 3:06pm.

Hayley wrote: "How does Subversion cater for large development projects where you have multiple developers changing the same file at the same time and frequently perform deliveries and rebases (more than twice daily)? We also often deliver between UCM projects so the changes can be reflected in one of the later releases of the project - can it cope with this?"

The most accurate answer would be "differently" ;-)

Even in UCM, if you're rebasing and delivering "more than twice daily," it may be that the branch-centric UCM model is getting in your way, and a bit of flattening might be in order. UCM's reflex is sometimes a bit over-eager to branch. If various developers need to see each others' work twice a day, they might ought to be on the same branch, directly, and let the working copy (or "snapshot view") constitute the personal sandbox. That would be the Subversion reflex, at any rate.

Cristian C's picture
Cristian C replied on February 7, 2011 - 9:33pm.

"dynamic view experience performance issues due to the network latency" - do you mean you are using dynamic views over a WAN? If that is the case, have you been exploring multisite? Or the web client?
Dynamic views over a WAN is a network killer.
If the management wants to cross to subversion, and hope that by pure magic UCM gets transplanted into subversion, then you all have to rethink the CM plans and revisit your current practices.
Cristian

CMCrossroads is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.