Our office will be closed from Tuesday, August 24 2010, at noon US Pacific time until Friday morning, August 27. We will not have internet connection and will not be able to check email.
Arnor Baldvinsson
Author: Arnor
Why, oh why do I want version control?
Version control systems have a reputation, at least in my mind, for being complicated, difficult to manage, difficult to implement, difficult to use, the list of difficulties goes on! So why on earth would I want to implement this stuff?
One of the most frustrating parts of development for me has always been the risk of breaking something, and spending hours rolling back and sometimes not be able to completely roll back. Then you have to restore from backup, and lose hours or even days of work - because at some junction in the development design I took the wrong turn for whatever reason (I'm sure I'm the only one who ever does that;). Then I find that I ended up on a dead end street going nowhere.
A well implemented and well functioning version control system can completely eliminate this risk. It will allow you to go digging into whatever blind alley of programming that you want, and make sure that you can come back to the real world without a scratch.
Many developers look at version control as being only for developer teams, where more than one developer is working on the same piece of program at the same time. While most good version control systems support exactly that, that is not their primary goal. Their primary goal is to make sure that you can, without risk, save your work and create copies of your work and roll back to any revision that you want, without losing anything. They enable you to pull out a two year old source file to check something, or pull out a whole system from last year, compile it and run it to see how your project looked a hundred builds back.
I currently have 20 products that I'm putting into version control (no, I didn't quite finish getting them all set up over the weekend). That is a total of 25,627 files in 2,599 folders! They include all the current Icetips products, some old products that I have not worked on for quite some time, and some new ones that are in development. I also have some in-house projects in there, too, that I work on from time to time and want to have the freedom to experiment on. With the safety net of version control, I am protected from myself. 😉
That is exactly what version control is about for me: Freedom to experiment without risk. Version control allows me to commit my project, create a branch and experiment on that branch. Note that Subversion/TortoiseSVN uses the term "commit" for saving the local files to the repository, which is the database with all the projects and all the revisions of the projects. Other version control systems also use the term "check in" about committing changes to the repository.
For example, I have been experimenting with some code for our OutlookBar. To do that, I make sure that my OutlookBar product folder is committed to the repository and then I right click on it and select "TortoiseSVN | Branch/Tag..." What this does is make a copy of my current project into a "branch". See the screen shot above.
I think a small tutorial on terminology is in order before I go any further. From the start you will hear references to Trunk, Branches and Tags. It took me a while to figure out what those things were and how they worked. Not necessarily because it was complicated, but just about every reference to those terms that I found did not explain them at all or did not explain if they are something that you create or the version control system should create for you.
The Trunk, Branches and Tags are simply folders inside the version control that make it easier to deal with versioning. If you are using Subversion/TortoiseSVN, it is up to you to create those folders in the root of the repository. See the screen shot above with the structures I have now set up for Build Automator. I have not created any Tags or Branches yet, so those folders are empty. When you create the repository, you should create those 3 folders.
The Trunk folder is where your current development version resides. This is what will become your next public release. This is where you copy files from to tags and branches.
The Branch folder is where you create branches of the trunk to work on without disturbing your trunk. This is where you create your experimental code, try things out, play like a pig in a pool of mud, do what ever you want to do without risking your precious code in the trunk! You simply copy your trunk there by using the "TortoiseSVN | Branch/Tag..." menu option. Use a descriptive name, like "Testing sub-classing in WindowsClass" for the branch.
The Tags folder is for your builds when they are done. Once you create a new build (hopefully using Build Automator and SetupBuilder:)) this is where you copy your trunk using the "TortoiseSVN | Branch/Tag..." menu option. Just like for the branches, you just pick the Tags folder instead of the Branches. As with the branches you should use a descriptive name for your tags, like "Build 1.2.1234, 2010-08-23" for a tag to give a good indication of what it is.
So in essence the tags are what you had, the trunk is what you have and the branches are what you will have! Actually pretty simple when you think about it! Once you have created the branch, you check it out to your working folder and work with it and commit to it. When you are done with your experiments, and if everything has gone well and you are satisfied that your new code is good, you can then merge your changes back to your trunk or basically copy the branch back to the trunk.
TortoiseSVN provides merge tools that can merge code for you and show you differences between files. It cannot merge binary files, so you cannot merge Clarion app files. To work with Clarion apps, you must work with TXA files and export them and import them to Clarion. For what I need, I'm not going that far at this time (I probably will sooner than later, though). For more information about how to deal with TXA and version control I strongly suggest you watch Rick Martin's webinars from Clarion Live. See my first blog about the version control for links to the webinars.
I will be away for most of this week and will resume my blog on Friday, August 27. Tomorrow I am going to Redmond for a .NET user group meeting where they will be demonstrating the new Microsoft LightSwitch - and the LightSwitch project manager will be there. Tuesday through Thursday the whole clan is going camping for the first time! For those of you who like photos and don't follow me on Facebook, please take a look at my online photos at http://arnor.zenfolio.com/ I will post some photos there when I get back!
Arnor Baldvinsson
Version Control, here I come!
Success! I have version control successfully implemented and have set up two of my largest projects, Build Automator and a client project. I will spend the weekend setting up the rest of my products, which are simple compared to the compiled projects, and get more familiar with TortoiseSVN.
I ended up uninstalling the VisualSVN Server and I'm only using the SVN Server. I had originally decided to put my development environment into version control, but ended up deciding against it. I have been using two builds of Clarion 6.3, 9053 and 9058. The reason for 9053 was a bug in some of the SQL drivers in 9058 that caused it to crash if file structures with MEMO files where used. For that I used the ClaSwitch program from Lee White at Clarion add-ons. It is a fantastic program and 
made it possible for me to deal with this - before I was using .bat files to handle this! Ugh!!! The program, just like my bat files, renames folder to "Clarion6". In my case I had Clarion63-9053 and Clarion63-9058. ClaSwitch would then rename the selected one to Clarion6 and run Clarion. This combination, I found out was just too risky as you might easily check out the wrong version into the Clarion 6 folder and mess things up or check out 9053, later run ClaSwitch and switch to 9058 and commit and now your 9053 repository has got files from 9058 and you really have a real problem on your hands!
In fact, that was exactly what happened today, and I ended up corrupting my 9053 install beyond fix (I don't think it helped that I was up to past 4am this morning working on this and back at it less than 6 hours later;), so I just took the plunge and moved everything to 9058. I had removed memos from all SQL tables as I did not need them in the Clarion program at all. Everything is working now, with just 9058 and I can move on:)
I did discover one interesting thing today in regards to adding folders to the ignore list. What I discovered is that the list is case sensitive. For example, the folder names I use for generated files are z-clw, z-inc, z-obj32, z-lib etc, but sometimes Clarion 6 decides to create them as upper case, i.e. Z-CLW, Z-INC, Z-OBJ32 or Z-LIB. After experimenting with my client project and compiling it I realized that the Clarion IDE was now creating the folders using upper case and the .clw and .inc 
files were being listed as un-versioned, when they should have been ignored. So I added Z-CLW and Z-INC to the list and the files disappeared from the list of un-versioned files!
Just adding files or folders to the ignore list is not enough. The files or folders also have to be deleted via the TortoiseSVN menus, see screenshot on the left. This removes them from the repository and once you commit the changes, the folders will be market as ignored. Note that Explorer does not always seem to update the icon overlays so the icons may indicate that the folders have been deleted and Explorer may even complain about them not existing if you select them and only them clean up after itself. There is a "Clean up" option on the menu which also refreshes the icon overlays.
So I'm happy with where I've got this, had hoped it wouldn't take this long, but I'm definitely getting there! I will spend the weekend setting up the rest of my product projects, all of which are considerably simpler than what I have set up so far, although some of them have a lot of test apps and projects that I do want to keep in there, but now I have a much better grip on how to ignore folders, etc.
Tomorrow I'm going to write up a bit about why I felt so compelled to dive into version control! Stay tuned and be good - I'm off to bed!
Arnor Baldvinsson
To SVN or not to SVN, that is the question!
One of my Clarion collegues, Brahn Partridge, pointed out to me this morning that VisualSVN Server is supposed to be used instead of the SVN Server, which was probably why it was not working for me at all. For now, however, I am going to use the SVN Server and get things working with it.
So if you are planning to set up a server on a network, rather than just use a local filesystem if you are working on a single computer, then you should pick either SVN Server or VisualSVN Server. With the SVN Server you connect to the repository with svn:// but with the VisualSVN Server you connect to the repository with either http:// or https:// depending on how you set it up.
One of the issues I have with TortoiseSVN is that it does not give you a list of the available repositories. So you have to remember what they are called (i.e. the folders in the repository root folder) in order to connect to them. Bit of a pain, but should get easier as I (hopfully) learn the names I used;)
Arnor Baldvinsson
Close but not quite…
After 25 years of working with computers and programming I really should know that things rarely, if ever, take the time that looks reasonable! No such luck, I still haven't got it;)
I started the day by sleeping in! Figured I might as well give my brain some rest before digging deep into version control. It was nice, but in the end I'm not sure it helped much!;)
I decided to start by installing VisualSVN Server on the machine where I have the SVN server installed. It told me that it would install Apache, so I stopped it and made a copy of my Apache install and created a restore point in case it messed up my existing Apache install. Ran the install and it went through without any problems and I choose all the default settings. I was able to access my repository without any problems and everything looked very good.
I added a user, very originally called "Arnor" and life was good. That is until I ran TortoiseSVN and tried to create a folder in one of my repositories and got an error about not having access to do that. I checked the user authentication files that TortoiseSVN creates and realized that VisualSVN was not touching those files at all and whatever VisualSVN was doing, TortoiseSVN was completely ignoring. After some head scratching I found that VisualSVN was creating the user files in the root of the repository (see screenshot above) and they did not look like they even had the same format as the TortoiseSVN files in the "conf" folder under each repository, see screenshot below.
After some more head scratching I decided to just skip this user authentication stuff in VisualSVN and copy the authentication files (see screenshot on the right) from TortoiseSVN to all my repositories. Growing up with DOS and BATch files came in handy when copying those files into 16 other repositories!
This got me going so I could now connect properly to my repositories and start messing with TortoiseSVN. I've spent the rest of the day trying to come to grips with TortoiseSVN. One of the design questions is about files and folders that perhaps should be ignored. For example all generated files are placed into subfolders, such as z-clw, z-inc, z-obj32, z-lib etc. This makes it very easy to do a clean compile, just delete all files from those folders and compile! But the question is should they be excluded or not? I'm not 100% sure yet. There are certainly advantages in keeping the generated source files (.clw and .inc) in case of an emergency corruption or something like that. But on the other hand rebuilding an application from clw and inc files would be a heck of a lot of work - besides I do have backups of the original sources outside of the version control. So I'm leaning toward ignoring all generated files.
Setting up the ignore list in TortoiseSVN is confusing. You can set it via properties and use the svn:ignore and check the "Apply property recursively" but it doesn't seem to work recursively on folders or, more likely, I'm not finding the right combination of folder name/wildcards or the right syntax. It appears that forward slash should be used in paths in the ignore list rather than backslash, but it doesn't seem to make any difference for me. Files in those folders are still versioned. For now I will just use the manual method. It is painful and it is easy to miss folders of test apps, but so be it. I can't spend days on chasing down information on software that is not well documented to start with, although it has to be said that TortoiseSVN has better documentation than just about any other open source project I have tried - one reason why I shy away from open source projects.
So, all counted, I'm getting there but by no means done! I've given myself until Monday morning to get this set up and done. I'm really glad I'm messing with this on a virtual machine where I can't mess anything up:)
Monday afternoon I'm heading to Redmond for a .NET user group meeting where John Stallo, who is a Lead Program Manager on the Visual Studio LightSwitch product team, will talk about LightSwitch which is being released into first beta on Monday! I'm excited about this product and think it may be what I had hoped that Clarion.NET would be.
Arnor Baldvinsson
Version control getting closer!
After spending couple of days arranging over thirty thousand files into the new folder structure, I spent today going throug Build Automator scripts and editor projects to change paths from the old source paths to the new ones. Tedious, error prone and utterly boring but had to be done!
Tomorrow I will finally get to start setting up the version control.
I'm using the latest builds of Subversion and TortoiseSVN. Yesterday I got a suggestion about using VisualSVN Server as management program for the Subversion repositories. Installing VisualSVN will probably be the first thing I do tomorrow to see how it works and then move on to get the repositories set up. Then it's on to finish setting up the repositories (I have most of them set up already) and get this thing all working nice and smooth!
Once I have all of this up and running, I will do a series of blogs about how to set this up. I am NOT an expert on this and have been fortunate to have some people in the Clarion community help me out, most notably Rick Martin, who did a webinar on this very subject on June 5, 2009 at Clarion Live (View webinar) and another one on August 6, 2010 (View webinar) Rick gave me some good ideas on how to set up the folder structure and I'm very grateful for that!
Why do I need version control? Good question! Stay tuned for the answer:)
Arnor Baldvinsson
LightSwitch Beta 1 released to MSDN subscribers
Microsoft has just released the first beta build of Visual Studio LightSwitch to MSDN subscribers. You can find it at the MSDN download site under the "New Downloads" section and then under "Visual Studio 2010". The full DVD image download is 507MB.
For more information about LightSwitch visit the Microsoft LightSwitch page. Watch the keynote presentation where, Jason Zander, Corporate Vice President, Visual Studio, announces a new addition to the Visual Studio product line. There are also couple of other videos that go a bit further demonstrating the product, but they all deal with exactly the same scenario so they repeat a lot of stuff.
LightSwitch is a new developer tool that makes creating business applications easy. In many ways it is like Clarion for SilverLight. We are looking forward to taking a good look at this too during the beta cycle and I am downloading beta 1 as I write this!
Arnor Baldvinsson
Redirection fix
Apparently my brain kept working on the redirection problem that I blogged about yesterday as when I woke up the morning I had a good clue what the problem was: Long path names. That turned out to be at least big part of the problem. However, it was not quite that simple, it never is! I didn't have time to do a complete research into the problem and nail down everything, but here is my findings and theories.
It appears that the compiler has issues finding files if the path to them includes pathnames that are more than 8 characters long. For example:
C:\Dev\Prod\Utilities\Src
causes problems, but
C:\DEV\PROD\UTILIT~1\SRC
does not cause a problem. But like I said, that is just part of it, as:
C:\Dev\Prod\XPTheme\Src
also caused problem. Either the problem is that there were other paths in the *.INC
line that use long path names (and there were) OR the compiler didn't grasp somehow that
C:\DEV\PROD\XPTHEME\SRC
is a short path name! I'm not sure which it is and didn't have time to figure it out.
Using a macro for any part of the path fixes the problem completely. For example, I created this macro:
DEV=C:\Dev
and then used
%DEV%\Prod\Utilities\Src
and there was no problem at all. So using the macros somehow makes the IDE correctly interpret the path name and probably convert it to a short path and the compiler is happy. When I had cleaned everything up, I ended up with just two macros being defined in my C60EE.ini:
Prod=C:\Dev\Prod Dev=C:\Dev
I also came to a final consensus on the folder structures I'm using, which seems to be working pretty well so far. The Dev folder has 3 folders now:
| C:\Dev\Prod | Products. Each product has its own subfolder so it can be easily added to version control. This way the entire product folder can be checked out and committed. |
| C:\Dev\Projects | Non product projects. This folder contains client projects, in-house projects and anything that cannot be classified as a product, free or commercial. |
| C:\Dev\Source | Other sources. This is basically a catch all folder for anything that doesn't directly belong in either of the above. Currently it holds templates and classes that I use in some of my own apps, but are neither a product or a project on their own, nor do they belong to a specific product or project. |
With the structure in place I spent most of today copying and moving files from my old structure to the new one and test compiling various projects and products in both Clarion 6.3.9053 which I use for one client project, and Clarion 6.3.9058, which I use for pretty much everything else.
So tomorrow will be the day that I convert all my editor projects to work with the new folder structure as well as my Build Automator scripts. Then I will get this on to a virtual machine and test the structure with Subversion to make sure that it holds up for the version control!
Arnor Baldvinsson
Redirection problems
I have spent most of the last week or so getting Subversion version control and TortoiseSVN up and running on our network. I will blog about that experience later, but to cut a long story short I have come to the realization that I have to change the folder structure that I use for my projects and products. They just don't work well for version control at all!
For about 10 years or so I have used a pretty good structure, that goes something like this:
C:\Clarion\Apps\C63\Products\Automator C:\Clarion\Apps\C63\Products\AutomatorDLL C:\Clarion\Apps\C63\Products\Utilities C:\Clarion\Clarion6 C:\Clarion\Shared\LibSrc\Automator C:\Clarion\Shared\LibSrc\Utilities C:\Clarion\Shared\Templates\Automator C:\Clarion\Shared\Templates\Utilities C:\Clarion\VC\Automator
Rather than being separated by products or projects, the folders are separated by over all contents, such as Apps, Shared, etc. Apps are then first separated by what version of Clarion they are being compiled in, then if they are projects (which are in-house projects), products or client projects.
The problem with this is that the separation of products is at the very lowest level of the folder structure, where as for version control, it should be as close to the top as possible so you can check out and commit the entire folder tree for all realted product or project files at the highest possible level. So I have decided on structure like this one:
C:\Clarion\Clarion6 C:\Dev\Prod\Automator\Apps\C63\Automator C:\Dev\Prod\Automator\Apps\C63\AutomatorDLL C:\Dev\Prod\Automator\Apps\VC\Automator C:\Dev\Prod\Automator\Src C:\Dev\Prod\Automator\Tpl C:\Dev\Prod\Utilities\Apps C:\Dev\Prod\Utilities\Src C:\Dev\Prod\Utilities\Tpl
This way I can for example check Build Automator out at C:\Dev\Prod and it will work with everything related to that product.
This of course requires a lot of changes to all sorts of files, such as the redirection files (RED files), Build Automator scripts, Editor projects, etc. Definitely not a walk in the park and as I'm kind of feeling my way around this it's very, very different from the way I have worked for many years and the older the dog, the slower it is to pick up new tricks:)
I expected the RED file to be relatively easy, but it has proved to be one of the most challenging; for a reason I'm not yet sure of.
Clarion allows you to set up "macros" in the C60EE.INI file as shortcuts to the folders. This is done in the [Redirection Macros] section in the INI file like this:
Shared=C:\Clarion\Shared SharedTpl=C:\Clarion\Shared\Templates SharedImg=C:\Clarion\Shared\Images SharedIco=C:\Clarion\Shared\Icons SH_Automator=C:\Clarion\Shared\LibSrc\Automator SH_Catalyst=C:\Clarion\Shared\LibSrc\Catalyst SH_CCS=C:\Clarion\Shared\LibSrc\CCS SH_Files=C:\Clarion\Shared\LibSrc\Filesandstuff SH_Magic=C:\Clarion\Shared\LibSrc\Magic SH_Other=C:\Clarion\Shared\LibSrc\Other SH_Previewer=C:\Clarion\Shared\LibSrc\Previewer SH_Utilities=C:\Clarion\Shared\LibSrc\Utilities SH_Xplore=C:\Clarion\Shared\LibSrc\Xplore SH_LibSrc=C:\Clarion\Shared\LibSrc
My corresponding Redirection file entries looked like this:
*.clw=.\Z-CLW ; .; .\CLASSFILES; %ROOT%\LIBSRC; %ROOT%\3RDPARTY\LIBSRC;%SH_AUTOMATOR%; %SH_CATALYST%; %SH_CCS%; %SH_FILES%; %SH_MAGIC%; %SH_OTHER%; %SH_PREVIEWER%; %SH_UTILITIES%; %SH_XPLORE%; %SH_LibSrc%\PowerOffice *.inc=.\Z-INC ; .\CLASSFILES; %ROOT%\LIBSRC; %ROOT%\3RDPARTY\LIBSRC;%SH_AUTOMATOR%; %SH_CATALYST%; %SH_CCS%; %SH_FILES%; %SH_MAGIC%; %SH_OTHER%; %SH_PREVIEWER%; %SH_UTILITIES%; %SH_XPLORE%; %SH_LibSrc%\PowerOffice
This works great and no problems with this at all.
But what I found today was that my changes for *.clw files and *.equ files (Equate files) as well as *.trn seem to work just great, the same cannot be said for the entry for the *.inc files! Currently I have this in my new Redirection file:
*.clw=.\Z-CLW ; .; .\CLASSFILES; %ROOT%\LIBSRC; %ROOT%\3RDPARTY\LIBSRC; C:\Dev\Prod\Automator\Src; C:\Dev\Prod\Utilities\src; C:\Dev\Prod\Checkbf\src; C:\Dev\Prod\ITMB\src; C:\Dev\Prod\ITME\src; C:\Dev\Prod\ITML\src; C:\Dev\Prod\Outlookbar\src; C:\Dev\Prod\PowerToolbar\src;C:\Dev\Prod\Previewer\src; C:\Dev\Prod\SQLBrowse\src; C:\Dev\Prod\Taskpanel\src; C:\Dev\Prod\WinFix\src; C:\Dev\Prod\XPlore\src; C:\Dev\Prod\XPTheme\src *.inc=.\Z-INC ; .; .\CLASSFILES; %ROOT%\LIBSRC; %ROOT%\3RDPARTY\LIBSRC; C:\Dev\Prod\Automator\Src; C:\Dev\Prod\Utilities\src; C:\Dev\Prod\XPTheme\src
This works without any issues for the *.clw (as well as the *.equ files, which I'm not showing here) but not so for the *.inc files! The compiler simply will not find *.inc files in the Automator, Utilities and XPTheme folders (there are several more, but I'm concentrating on just those to start with).
However, if I create a macro for the C:\Dev\Prod\Automator\Src and C:\Dev\Prod\Utilities\src paths, and use that macro in the Redirection file, everything compiles without a problem! I.e.
[Redirection Macros] BA=C:\Dev\Prod\Automator UT=C:\Dev\Prod\Utilities *.inc=.\Z-INC ; .; .\CLASSFILES; %ROOT%\LIBSRC; %ROOT%\3RDPARTY\LIBSRC; %BA%\Src; %UT%\Src;
Since I had set up macros for all the *.inc folders last time I guess I never ran into this anomaly before, or maybe that was why I had created all those macros - just can't remember!
Bottom line is that if something weird happens in your redirection file and it doesn't seem to work, try to create a macro in the INI file and see if that helps! Note, that in order to create the macros, you MUST close the Clarion IDE! In the INI the macro does NOT include the %, i.e. BA defined in the INI becomes %BA% when used in the Redirection file.
Arnor Baldvinsson
Books that I’m reading
I was in Barnes and Noble today and bought two books that looked interesting.
The first one is Seth Godin's book "Purple Cow" It is about the importance of making your company or product stand out in the marketplace, just like a purple cow would do in a field of green grass and a flock of brown cows. No matter what line of work you are in, some form of marketing is something that we all do. There are people who are natuarlly good at it and then there is the rest of us who have to work at it! This book has 218 reviews at Amazon.com and get's mixed reviews. Some readers feel it doesn't really deliver much substance and is more a book written to show how smart the author is rather than to show us how to learn from his smartness, which I guess kind of makes him a smartass;) The edition that I have was published in 2009, but I'm not sure when this book was first published. The oldest reviews on Amazon.com are all from May 8, 2003.
The second book is called "Your Virtual Success: Finding Profitability in an Online World" by Alan Blume. This book is about virtual businesses and how to make them work. It is also a lot about marketing. Published on June 20, 2010, this book gets solid 5 stars at Amazon, but only 4 people have reviewed it so far.
I have only just started looking at them and will write about what I thought of them once I'm done reading them:)
My bedside reading these days are some of Patricia Cornwell's Scarpetta books.
Arnor Baldvinsson