Windows 7 / Windows Vista Virtual Store Bug

In previous blog postings here I have talked about Windows 7 - as compared to Vista and XP - and all-in-all, (IMHO), Windows 7 is a pretty slick operating system.

Not unlike other "new" operating system releases, there are - and will be - teething problems that will eventually be worked out.

There IS one issue that - though probably "by design" - is sufficiently evil that it borders on being a really nasty bug: Neither Vista nor Windows 7 refresh the Virtual Store's image of a file when the target file changes.

At this point you may be saying "Virtual Store"??! What the. . . .!!!

A little bit of an explanation is in order:

Vista was Microsoft's first real serious attempt to tighten up their security model and they did this in several different ways . . .

• They implemented the now infamous UAC - User Access Control, (or User Account Control depending on who you're listening to).
• They designated several important areas within the operating system as "protected" areas - you are unable to write to these areas without specially elevated privilege levels.
• However - to maintain backward compatibility with older programs written for XP - they allowed a special exception: If a non-privileged process attempted to access a privileged (protected) area, a write-enabled copy would be created that would be accessed as if it were the original.
• They way they accomplished this is by using a special directory called the Virtual Store.
• This directory is located at:
  C:\Users\[current user]\AppData\Local\VirtualStore
  and it contains an image (copy) of any protected items that were accessed by non-privileged processes.

The idea has merit. Older programs written for XP or before, that automatically assume that EVERYONE is an Administrator level user and write all over the system willy-nilly will be given access to the Virtual Store instead of the Real Application. Virii who wish to install themselves into the Program Files directory get side-tracked to the Virtual Store - or raise a UAC - instead of just sneaking in.

The Virtual Store operates according to some very simple rules:

• If a non-privileged application attempts access to a privileged area (C:\Program Files\MyApp, for example), a copy of the thing they tried to access is copied to the Virtual Store.
  Viz. It would be copied to:
  C:\Users\[current user]\AppData\Local\VirtualStore\Program Files\MyApp
  . . .and the application would be given access to the Virtual Store instead of the real program path.
• If a non-privileged application tries to access a protected area, AND there is already an image of the object requested in the Virtual Store - the application is automagically re-directed to the copy instead of the original. In other words, if C:\Users\[current user]\AppData\Local\VirtualStore\Program Files\MyApp already exists (in the Virtual Store), then any reference to C:\Program Files\MyApp will automatically be redirected to the copy in the Virtual Store - AND - as far as the executing process is concerned, it will actually appear to be executing out of the C:\Program Files\MyApp directory!
• Pivileged applications - running as true administrator equivalents via the UAC or by inheriting elevated privilege - always get access to the real file, not the copy in the Virtual Store.

 This is all well-and-good, but if you're paying attention, you will have noticed a subtle issue has crept into the system:

• If I publish a periodic (or bug fix) update to MyApp\. . . . the updater or .MSI file that does the updating will actually and truly write to the real Program Files folder.
• However, once the update is done and the non-privileged application starts to run again, it only sees the cached version of the file in the Virtual Store's folders.
  
  - AND -
  
  
• The presence of an updated version in the Real Folder does NOT trigger a refresh of the cached data in the Virtual Store! 

There's the problem. Stale data in the Virtual Store will cause your program to run amok, even after a patch or bug-fix has been installed.

The solution is simple:

• Never store data or user-generated configuration files in a protected area, such as Program Files.
• When the non-privileged application begins to run - before it does ANYTHING else - it should check the Virtual Store manually - via the explicit path - and see if there is a "MyApp" directory there.
• If there is a virtualized copy there, delete it.
• Installer software or .MSI updaters should follow the same rule when they begin running.

This will force Windows to update the Virtual Store so that when the non-privileged application runs - and accesses data in the Virtual Store - that it will always get the latest-and-greatest version.

Note that this workaround will also work on a legacy system running XP or earlier - since the path to the Virtual Store will NEVER exist on these earlier systems, the program's logic will simply skip past that and go on to launching the application.

What say ye?

Jim

How to migrate Outlook mail from XP to Windows 7

One of the issues that's sure to come up, now that Windows 7 is gaining traction, is how to take your Outlook settings, e-mails, and such-like and move them to a Windows 7 machine.

---

 In my own case - I have Outlook 2003 installed on Windows XP and I am building up a "new" machine with Windows 7 on it, so I needed to figure out how to migrate my stuff.

Also - I tend to swap computers back-and-forth.  For example, when I travel, I take a Compaq laptop with me, but I use my AthlonXP powered desktop when I'm at home.  So, I periodically "syncronize" (swap back-and-forth) the e-mail files between the two machines.

In XP the correspondence is relatively easy to follow, since most of the Outlook data is located in two places:

1. C:\Documents and Settings\[user name]\Application Data\Microsoft\
   
   There is an "Outlook" folder that you copy in its entirety to the new location.  This folder has stored settings specific to your e-mail stuff.
   
   There are also "Signature" and "Stationary" folders here as well, if you have pre-defined signatures or stationary you use.
   
   
   
2. C:\Documents and Settings\[user name]\Local Settings\Application Data\Microsoft\
   
   There is an "Outlook" folder that you also copy in it's entirety.  This folder actually contains your mail files and such.  Depending on how much e-mail you have, these files can get pretty darn big!
   
   
   
3. Certain other data about your e-mail - such as account information, logins, server preferences, etc. - are (supposedly) located somewhere in the Registry.  I haven't gone looking for them yet.  And to be perfectly honest, I'm not sure I want to just go cutting, hacking, and pasting from XP's Registry into Windows 7's Registry.  I'd really want to know a whole lot more about what I'm doing before I start that kind of Open Heart Surgery on a computer.

Note:

• I am talking about Outlook - not Outlook Express.
• I've never used Outlook Express, so I cannot comment on it.
  
  
• I'm talking about Outlook 2003.
• If you have Outlook 2007, I have heard rumors that it is essentially similar.  I haven't installed Outlook 2007 yet, so I can't speak directly to that from my own experience.
• If you have an older version of Outlook, you will have to first upgde to 2003 (or better) before you can even think of doing any of this stuff!
  
  
• I am talking about a personal installation of Outlook, using ISP provided mail servers - usually POP and/or SMTP mail servers.
• If you have Outlook set up to use an Exchange Server for your e-mail, you probably have a system administrator that should be helping you with this.  Migrating Exchange hosted e-mail is outside the scope of this article.  It's not difficult, it's just a totally different beast.
• Sysadmins:  The user's localized settings and .ost /.pst files might well be located in these same places, based on the installation's configuration.
  
  
• I am talking about a manual migration, dragging files and folders around "brute-force", so to speak.
• The good part about this is that it costs you absolutely nothing, whereas the payware migration software out there can hit you up for $60 or better.
• The bad part about this is that - bottom line - YOU own it.  At the risk of mangling a great line from Blazing Saddles:  "One false move, and the e-mail gets it!"  The bottom line here is that if you know enough to do this, you also know enough to not come crying to me if you accidentally clobber your e-mail.  Can YOU say "backups"?  Ahhh!  I KNEW you could!
• If you have a LOT of machines to migrate, (and if you have that many, you should really consider a centralized mail-store like Exchange), the payware solution might end up saving you money and grief in the long-run since they supposedly migrate everything - including registry settings, etc.
  
  
• You will have to have "show hidden files and folders" as well as "show hidden system files" enabled on both XP and Win-7 before you will see these paths. 

---

 Migrating Outlook from Windows XP to Windows-7 Executive Summary:

The basic folder-to-folder correspondence is as follows:

The XP folder:
C:\Documents and Settings\[user name]\Application Data\Microsoft\Outlook\ (as well as ..\Signatures\ and ..\Stationary\)

Is located in the Windows 7 folder:
C:\Users\[user name]\AppData\Roaming\Microsoft\Outlook\ (as well as ..\Signatures\ and ..\Stationary\)

==========

The XP folder:
C:\Documents and Settings\[user name]\Local Settings\Application Data\Microsoft\Outlook\ (where all the mail files reside)

Is located in the Windows 7 folder:
C:\Users\[user name]\AppData\Local\Microsoft\Outlook\

---

 How I accomplished the move:

1.  The first thing I did was to install Office 2003 (which included Outlook in my case) on the target Windows 7 machine.
   
   
2. Once the installation completed, I shut down the computer and attached the old "XP" drive as secondary.  You can attach it directly to the hard drive controller - (I have several drive-bays on my main machine for just this reason) - or you can attach it through a hard-drive-to-USB adapter.
   
   
3. Once you restart the machine and launch Outlook, you are prompted to set up an e-mail account. I set up a bogus@bogus.com "fake" account so that Outlook would build the appropriate directory structures.
1. I didn't set up a REAL account yet, because Outlook gets pretty darn agressive when it comes to sucking e-mail off your servers - and I wanted to get the "where to put it" set up first.
   
   
4. I then closed down Outlook and opened up File Explorer windows in Windows 7 - one for each drive.
   
   
5. I copied the
   
   C:\Documents and Settings\[user name]\Application Data\Microsoft\Outlook\ (as well as ..\Signatures\ and ..\Stationary\)
   
   folders off of the XP drive and placed them in the
   
   C:\Users\[user name]\AppData\Roaming\Microsoft\Outlook\ (as well as ..\Signatures\ and ..\Stationary\)
   
   folders on the Win-7 machine.
   
1. You will get some prompts to overwrite things - go ahead and let it.
   
   
6. I copied the
   
   C:\Documents and Settings\[user name]\Local Settings\Application Data\Microsoft\Outlook\
   
   folder off of the XP drive and placed it in the
   
   C:\Users\[user name]\AppData\Local\Microsoft\Outlook\
   
   folder on the Win-7 machine.
   
   
   
1. You will get some prompts to overwrite things - like Outlook.pst, or Desktop.ini - go ahead and let it overwrite.
2. Because you are moving your main mail files here, this copy may take a LONG time to complete.  Maybe even measured in hours.  Go get a cuppa coffee.  Read the paper.  Take a nice loooong walk outside.  You get the idea, right?
   
   
7. Once the copies are complete, shut down and disconnect the XP drive - just in case!
   
   
8. Restart the computer, and restart Outlook.
   
   
   
   You should notice that Outlook comes up with all your mail folders intact.  If this does *NOT* happen, stop right here and review your steps.  This has to be right before you can move on.  Hopefully you copied from XP to 7, and not vice-versa, as that will have clobbered the mail-files on the XP box.  This is why backups are so important!
   
   
   
9.  Now, you need to go to "Tools --> E-Mail Accounts" and begin to configure your actual e-mail accounts, both send and receive, just the way you had them before.  Passwords, server names, the works!  (This is the stuff in the Registry that we didn't hack with, remember?)
   
   
10. When you are setting up your e-mail account, or accounts if you have more than one, I usually go to the "More Settings" button to open up an advanced settings window, and then select the "Advanced" tab.  On this tab you will find the check-box "Leave e-mail on server", and check this box to enable that feature
1. I usually check "Leave e-mail on server" to "ON" when I'm testing a new e-mail client as a saftey precaution.  This way if I accidentally screw something up, I can always go back to the old client on the old machine and everything's still working just fine!

---

Caveats:

Because we did not even try to mess with Outlook's Registry settings - certain information and preferences does not automatically come across when the move is made.  However these things can be fixed-up rather quickly.

• In my case, I have a great many individual mail folders - and sub-folders! - along with a whole host of rules set up to automagically sort my e-mail upon arrival.
• Though the rules - and their associated folders - survived the migration, sometimes the correspondence between the folder within a rule - and the actual mail folder itself - gets un-linked.  Probably because of the big differences in paths between the two systems.
• What will happen is that you will generate Rule Errors complaining that it could not move the mail to the respective folder - and the affected e-mails will remain in your Inbox instead of being moved to the correct folder.
• To fix this, go to the  "Tools" menu, select "Rules and Alerts", and then on the "E-mail Rules" tab, select the rule that had the problem.  Double click on the underlined folder name Outlook could not find, (it may appear to be absolutely correct!), and re-select the corresponding mail folder from the list.  Click "OK" to close the rule and save changes.  Next click "Run Rules Now" along the top, select the rule you just edited, and run that rule on the Inbox.  All the offending e-mails should now get moved to the correct folder.
• Repeat this for any other rule errors encountered.
  
  
• The Outlook "reading" format will default back to the original screen layout.  In my case, I greatly prefer the "reading pane on the bottom" layout, so I go to "View" on the top menu bar, select "Reading Pane" about half-way down, and then check "Bottom".
  
  
• If you moved your Signatures folder, all your custom signatures will have made the move with you, but the associations between the signatures and your e-mails may not have.
• To fix this, go to "Tools" on the top menu bar, then select "Options" (all the way down at the bottom!), and then select the "Mail Format" tab.
• Down at the bottom of this tab are the fields where you set the signature for new mails as well as replies.  If you have more than one e-mail account, you have to select each one individually from the "Select signatures for account:" drop down, and set signature preferences for each one.
• Select "OK" to exit.
  
  
• You may notice that Outlook now starts out on the topmost, most global, folder view called "Today".  Usually people have Outlook configured to start with their Inbox showing, (or their Calendar, etc.) rather than the, (IMHO, totally useless), "Today" page.
• To fix this, if you have the "Tools" --> "Options" page still open, click on the "Other" tab, and then near the top click on the "Advanced Options" button.  Otherwise, navigate back to it, click on "Other" then "Advanced Options".
• At the very top of the Advanced Options page is a browse selector labeled "Startup in this folder:".  Click on "Browse" and select the Outlook folder you wish to open when you start Outlook.
• Click on the "OK" button twice and you're done!
  

What say ye?

Jim

Windows 7 - Patience is Rewarded - Drivers

I've already taken the time to review Windows 7 and my general impression was that it was not only much better than Vista ever hoped to be, but it could be considered a worthy successor to the venerable Windows XP.

---

However - as with any shiny new toy - there is a bit of a dark spot to contend with, and that's finding drivers for stuff more than a very few years old.

Example:

• My wonderful Sound Blaster Live! card that I bought a while back - and paid hefty bucks for! - was declared "end-of-life'd" by Creative and they said they'd not support it for Vista - OR Windows 7.
• NVIDIA didn't even want to consider looking at drivers for my GFX - (7800) video card.  I wasn't too disapointed there, Vista (and 7) picked it up nicely.
• My Microsoft fingerprint reader(s) - apparently there was a licensing scuffle between M$ and the folks at Digital Persona - who are the actual end-product manufacturers - and M$ could not get licensing for either Vista or 7. . .  This really broke my heart because I have adopted the Fingerprint Reader extensively - I own about seven of the beasties! - and I use them just about everywhere.

And as I continue trying to build up a Windows 7 system that mimics my fully-tricked-out XP install, I am sure that I will find other issues.

Lest anyone get the impression that Windows 7 is a real headache when it comes to drivers, let me assure you that - in my case - about 90% of the stuff I had drivers for fell into one of three broad categories:

• Windows 7 picked it right up and it didn't need additional driver support - or it picked it up well enough so that Windows Update could provide a decent driver.
• A web search at the manufacturer's site showed a driver (or program version) that would work with Windows 7.
• The pre-existing driver for XP installed without a hitch.

However - the few items that I could not find drivers for, were a big dissapointment.

---

Fear Not!

Apparently Windows 7 is doing so well - compared to the debacle that was Vista - that manufacturers are having second and third thoughts about withholding driver support!

Case in point:

• Today, I just discovered that Creative has released Sound Blaster Live! drivers for Windows 7.  I don't know that they were released today, but I found them today. . . .
  (I can't wait to remove the CRAPPY SIIG SoundWave 5.1 PCI card I bought and slide that Sound Blaster back in there - the difference in playback quality between the two is immense.  Unfortunately not in favor of the SIIG card.)
• NVIDIA has released to Microsoft - for inclusion in their Windows Update packages - updated drivers for the 7800 series video cards.

Additionally - and this is something that was conspicuous by its absence with Vista - I'm beginning to see realistic workarounds being developed by outside users for those things that won't be able to get support - like my fingerprint reader!  It turns out that there are a few hoops you have to jump through - certain ones deliberately placed in Windows 7 by M$ because of the licensing issues I mentioned - but having done so, you have a wonderfully working fingerprint reader.

Likewise, I'm seeing workarounds for things like my ancient HP ScanJet 3300C - a wonderful scanner, it would be a pity to have to trash it simply because HP won't support it on "7"  (Which, IMHO, is an absolute disgrace - even if there were something "un-official" that could be done to help people out, it would be a lot better than having to shell-out for all new hardware devices just because you swapped operating systems.)

---

There is one big cavaet here - most of this applies ONLY TO 32 BIT VERSIONS of Windows 7.  I have read repeated reports of the 64 bit versions not accepting the driver workarounds.  Which is a real crying shame.  In a way though, this doesn't surprise me.  I remember fussing with the 64 bit versions of XP, and getting driver support for it was - well, let's just say it was something of a challenge and leave it at that!! 

Maybe there are fundamental architecture differences that preclude 32 bit drivers from working in the 64 bit environment?  I remember back when Microsoft moved from Windows 3.n to '95, and then to '98.  Driver compatibility between the "old" 16 bit drivers and the "new" 32 bit operating systems was a real issue.  Some worked if you faked it, but many didn't, and there were numerous reports of people who crashed their systems trying to fit a 16 bit driver into a 32 bit slot.  I suspect we're going to see that same thing again - except (hopefully) by now process fault preclusion has advanced far enough to prevent an errant user from totally trashing their hard drive.

In any event - hopefully the manufacturers will have mercy on our miserable souls and will help us with - at the very least - some kind of workaround, even if it's "on the down-low", until this older hardware goes the way of the full-hight Rodime 20 meg MFM drives - and 8" floppies!

What say ye?

Jim

Windows 7 Easter Egg - "God Mode"

Windows 7 is out in dead earnest - and the Easter Egg Hunt has begun!

---

For those who may not be familiar with the term, an "Easter Egg" (within the context of a piece of software, or a game), is a "hidden" feature or capability that can only be accessed by knowing some (supposedly) secret command that unlocks the feature.  Most previous versions of Windows have had some kind of "Easter egg" or another - ranging from clever tricks to silly animations.

In some cases, Easter Eggs are clever "short-cut" features that are used during in-house development and testing to speed the development process - or make the developers / testers lives easier.  They might even be seperate "special" apps - that let the development teams do something clever and convenient.

---

I don't know this for a fact, but I suspect that a lot of the previous Windows Power Toys were just this kind of thing - Easter-egg kinds of links, or special clever helper apps - that eventually went main-stream.

The "new" God Mode in Windows 7 is just this kind of thing - it's a way to collapse all the features within Windows 7's various control panel features into one convenient folder.

Accessing this special "God Mode" is easy:

• Create a new folder somewhere.
• Change the name to GodMode.{ED7BA470-8E54-465E-825C-99712043E01C}

Once you do that the folder's icon will change into something like a control-panel and it will - automagically - contain dozens of icons to various system controls.

---

Note:  This information was originally posted on the CNET column "Beyond Binary" by Ina Fried (http://news.cnet.com/8301-13860_3-10423985-56.html) on Jan 4th, 2010.

We're Generating some Buzz!

OK - maybe a small humming noise? ( :-) )

I heard from a former colleague who had this to say about the QA Tech-Tips Blog:

---

Thank you again for the QA tech tips - they are very much appreciated.
I liked the site, I will re-read all of the tips soon - I did read most at the time you emailed them.
Just so you know, I have actually quoted the latest ("The case of the vanishing internet") in one of the bugs that I logged recently.

Keep the good info coming!

---

Ahhh! Nothing I like better than a little well-directed praise!

Seriously now - that's the entire point of these posts and this blog:  I like to mess around with stuff and poke my nose into the nooks, crannies, and dusty corners of technology.  And I believe what confuses me will likely confuse others.  What interests me - and piques my curiosity - will likely pique the curiosity of others.  What I believe to be important "real-world" information to me, may well be just as relevant and important to others too.

And that's the point.  As far as I am concerned, I have already done the head-banging, hair-pulling, voodo dances and sacrifice of small rodents to the Technology Gods to get the information I got.  Why should everyone else have to do the same head banging?  One concussion per group please!

I have been blessed with the opportunity to learn much from some of the most talented people I know.  So it is my pleasure to pass along what little I can to others as well.

"So it shall be written - so it shall be done!"
(Yul Brenner - The Ten Commandments)

What say ye?

Jim