This week so far has been a difficult one – too many late nights. A couple of holiday parties, meeting a good friend in Boston and then the headache caused by the OS upgrade to my machine, which I discuss in detail later. I am looking forward to the trip to Jupiter, FL next week with my family. my sons and I plan to play golf and hopefully get some time to relax too.

I was all excited by the prospect of upgrading to Lion sometime in October when I got the DVD. Here we are in mid-December and finally it happened. My frustration with the upgrade is caused by unique circumstances surrounding my own computer, however,  there have been more than the usual complaints about a Mac OS upgrade like the one discussed here.

I had turned on FileVault, a software that encrypts the home directories belonging to individual users on a Mac. This is a good practice to prevent against data theft in case the computer is lost. FileVault under Snow Leopard worked fine for me though there have been many reported issues of slowness. There also have been a lot of confusion around what exactly is encrypted. Apple document on this says it is the home folder of the user. If that is the case, I have some web applications and database passwords for MySQL on my local machine that are outside my home folder, so were they not encrypted? I could not get authoritative answers to these. But, the fact that my home folder was encrypted was a big relief. It is also required by the College’s Written Information Security Program.

BTW, FileVault II in Lion is supposedly better (meaning it is more efficient and faster) and does the whole disk encryption, rather than just the home directory.

Another point of confusion had to do with FileVault and Time Machine.  What really is stored in Time Machine when FileVault is turned on? I wish I had paid a lot more attention to these than I did. I assumed that since Time Machine kicks in AFTER I log in, by definition, my home folder should be unencrypted, so it got backed up that way.

So, in October, when I was ready to upgrade to Lion, I tried to do the first step – turn off FileVault. This attempt failed when the system complained that I did not have enough space on my disk. I found it hard to believe because I had over 150 GB left. After trying it several times, I simply gave up.

Then my colleagues here helped by installing a new larger hard drive with Lion installed on it. The plan was that I will go to my Time Machine and retrieve my data. Failure again. Time Machine complained that it was unable to access my home folders. Now, it sort of made sense that this has something to do with FileVault. Rather than persisting on this, I quickly gave up and get my old disk back up. The advice I received was that I should move some of my data to an external disk and then turn FileVault off. I didn’t want to do it because it was too much work. My rationale was that I believed that there was a lot of space available, secondly, FIleVault doesn’t even tell me how much space it is looking for, so how am I going to guess what is the right amount to delete?

Destined to be stuck with Snow Leopard forever and ever, I put away the Lion Disk. It was laughing at me everytime I looked – “LOSER!”.

Then, last Sunday, while bored, I said I am going to figure all of this out. So, I was getting ready to read up and solve this big mystery. But, I wanted to give one last try of turning FileVault off. And magically it worked. I swear that I deleted nothing, if anything, I added more. The only difference I can think of is that I did not have the Time machine disk attached.

I was so happy. However, my machine was unusable for hours at a stretch while this turning off process ran. On Sunday evening, I started the Lion install. Failure! It complained that my file system was not set up for journaling. OK, that was an easy fix. Started again and went to a holiday party. Got back to see Failure – why? Because now, Lion installer was complaining about not enough space for the creation of a recovery partition. So, Dear Apple, how much space do we need for all these? Can someone lay all of these out clearly in one place?

Frustrated again, I took the machine to my LTS colleagues. Now that FileValuts was off, they did their magic and handed me a larger disk (750GB; the older one was 450GB I think). I was so happy that this all worked out in the end. Except, my desktop didn’t come through, many apps like Apple Mail or Outlook were not finding some of my legacy emails. Soon it became apparent that about 100 GBs of data didn’t come through from the old to the new disk.

Not so concerned, I asked that I get the old disk be given to me so I can connect it and recover files. I began this around 8 PM on Tuesday night. It just didn’t go well. Despite the fact I was the superuser (root) and using all the necessary Unix commands from terminal to copy files over, they were failing. I finally traced the whole thing to file locking. As to why files were locked, I can only guess. All old files were owned by a user id 502 (my guess is that was assigned to my username in the old system). But, as a superuser, why could I not access it, I don’t know.

You don’t unlock files on a daily basis, so I hunted around and found the magic command uchg and removed locking and copied the files over and changed ownership in the new disk to my username. Then at 11:30 PM or so, I started the copy process for a 50GB folder and went to bed. When I got up at 5:00 to get ready to move to the next folder, I found out that the copy had failed. The external disk was unmounted. I thought I had done something stupid which loosened the connector. So I tightened everything, remounted the disk and started the process and it failed again – I heard a click and the disk shutting down. It hit me pretty bad that I may have a bad disk!

I was scared. I really was. If my original disk was damaged, how confident can I be that my Time Machine had all the files? After my FileVault Removal I kept getting complaints about not enough space on the Time Machine disk, could all these be related? Did I lose over 70GB of my important files, images and videos?

I quickly came to Clapp and hooked up my Time Machine volume. I was shocked that the latest backup I could go to was Monday! What happened to all those days of backing up? Was this all the result of FileVault being removed on Sunday? I had a Tim Tebow moment. I started praying to all the Hindu Gods I could remember the prayers for.

I found the missing folders in the latest backup in what seemed like an eternity. When I went to copy them over, it prompted me for the root password. I said Oh No! what is all this about? But when I gave the password, it began copying. I was sitting in my Ball Chair simply watching the progress of copying, not wanting to touch the machine for the fear that it may bring some bad luck.

After it copied, I was unable to access the files on my new disk. Again – 502, locking etc. I repeated the process of unlocking, changing ownership and all that. I started the restore of 50GB folder and went to the apartment, came back to see it had completed. I did the same for all other missing folders, rebooted. Just like magic, I had my cluttered Dock with 50 applications and the clean desktop appeared and all apps worked. I was golden.

Given that we are dealing with giga bytes of data, how can anyone be ever sure that all files were copied over? I did the best by comparing the source directory size and target directory size and they all look good. One of these days, I may find that some files are missing or corrupt. That is the name of the game.

I would say that this particular upgrade has been far less ideal than what we all have come to expect from Apple. Could it be because things are so complex these days that it is impossible to come up with anything smoother? Or is it that the fiercest advocate for consumers (Steve Jobs) is gone? Who knows.

Please don’t ask me what have I benefited from after the change. I am still looking for them. In fact things have degraded a bit because I am still getting used to some of the gestures acting in opposite directions (I know, I know, I can change these in settings). These are the same type of complaints that accompany any change – Email systems, or operating systems.

Yet again, Change is Constant and Painful! And I am reminded of a quote from a wise man “Why is it that all these upgrades turn out to be downgrades for me?”