Windows uses the Shell BAG data to arrange folders and files in Explorer views, as well as to monitor recently used and frequently used applications in the Start-Menu among other things. Forensic experts use it to find traces of deleted file/folders with associated MAC times and file paths. It even tracks network paths and paths on external thumb drives that were once plugged into the machine.
When I started this research, I didn’t know of any product nor any freeware or documentation that explains the BAG format, so I decided to reverse engineer it myself. However since, I have learnt of a
paper presented this year that talks about the BAG file and its usage to recreate user activity at the DFRWS conference. They have a tool called
TraceHunter, available on request only, but no available documentation on the format. I haven’t seen the tool yet.
Update: I have been informed by the folks at tracehunter that they will not be giving me a copy!
I have
read on a blog that a once freeware tool to accomplish this was bought by Paraben and incorporated into their product ‘Registry Analyzer’ which is now called P2 Commander ($129).
About the BAG format
It is prevalent in all windows versions since XP and is used in following registry keys:
- HKEY_CURRENT_USER\Software\Microsoft\Windows\Shell\Bags
- HKEY_CURRENT_USER\Software\Microsoft\Windows\Shell\BagMRU
- HKEY_CURRENT_USER\Software\Microsoft\Windows\ShellNoRoam\Bags
- HKEY_CURRENT_USER\Software\Microsoft\Windows\ShellNoRoam\BagMRU
- HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\StreamMRU
- HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\RecentDocs
Vista and Windows 7 have these additional locations (updated)
- HKEY_CURRENT_USER\Software\Classes\Local Settings\Software\Microsoft\Windows\Shell\Bags
- HKEY_CURRENT_USER\Software\Classes\Local Settings\Software\Microsoft\Windows\Shell\BagMRU
- HKEY_CURRENT_USER\Software\Classes\Wow6432Node\Local Settings\Software\Microsoft\Windows\Shell\Bags
- HKEY_CURRENT_USER\Software\Classes\Wow6432Node\Local Settings\Software\Microsoft\Windows\Shell\BagMRU
My Research
Very little has been known about the technical specifications of the Shell BAG format. I present my research and an Encase script to parse NTUSER.DAT hives and extract the data. The source code is a mess right now and many things are hard-coded, so I have not made it available yet.
I have been able to understand the basic format, although there seems to be quite a few variations of how it’s used in the different registry keys and at different levels, ie, it varies depending on whether the BAG artifact is from a file or folder, regular file or ZIP file, etc.. Sometimes GUIDs are used (instead of filenames) for well known items like Control Panel as well as application created shortcuts.
Structure Format
struct BagFile {
USHORT BagSize; // Length of BAG structure
USHORT flags;
DWORD size;
USHORT ModifiedDOSDATE; //Modified Date GMT
USHORT ModifiedDOSTIME; //Modified Time GMT
USHORT FlagUnknown;
char name[]; // (DOS short filename)
//extra byte here sometimes to align to even byte boundary
UnicodeBagData UnicodeData;
};
struct UnicodeBagData
{
USHORT LengthOfUnicodeStructure;
USHORT Short1; // 0x0003 for XP, 0x0008 for win7
USHORT Short2; // 0x0004
USHORT Short3; // 0xBEEF
USHORT CreatedDOSDATE; // Created Date GMT
USHORT CreatedDOSTIME; // Created Time GMT
USHORT AccessedDOSDATE; // Accessed Date GMT
USHORT AccessedDOSTIME; // Accessed Time GMT
DWORD Unknown; // usually xp = 0x14, win7 = 0x2A
// Vista and Windows 7 Extra Fields (22 bytes total)
DWORD MftFileId;
USHORT Unknown1;
USHORT MftSequence;
DWORD Unknown2;
DWORD Unknown3;
DWORD Unknown4;
USHORT Unknown5;
// END Vista extra fields
wchar name; // Unicode Filename
USHORT Unknown6;
};
Although this is the basic format, there are several variations depending on flag values. The elements are likely contained in more structures nested within, hence the variations.
In the StreamMRU values, many such BAG structures are chunked together into one reg value, which represents the full file path. For example, if a folder path is “C:\Program Files\Microsoft\Office”, the registry value will have the following data
[BagFile for C:] + [BagFile for Program Files] + [BagFile for Microsoft] + [BagFile for Office]
All will be appended into the same reg value.
Files vs. Folders
Since BAG data is mostly used by Explorer to remember view settings for each folder, we only find BAG entries for Folder data under ‘Software\Microsoft\Windows\Shell\BagMRU’. Each folder under that key has a NodeSlot value which represents the unique folder number for its settings which are stored at ‘Software\Microsoft\Windows\Shell\Bag’.
The keys under this key store both file and folder metadata. Any time you move an icon in explorer, its new position in that folder is tracked here. For each NodeSlot value in BagMRU, there is a folder with that number as name. Not all views are tracked, so you will see some empty folders under this key and some with many values. One of these values will look like “ItemPos 1024x768(1)” or whatever your screen resolution is at the time of the event. There can be multiple of these items for multiple monitors, change in screen resolution, etc.. The ItemPosxxx key consists of many BAG structures appended together. Unlike StreamMRU, these don’t represent a file path, they are simply metadata for all the files/folders in that particular folder that it represents.
BAG Data Organization
The data is organised exactly as it appears in Windows Explorer, with the desktop being the root folder of everything.
The paths inside BAG files will have many redundant entries pointing to the same object in a different view depending on whether you browsed to the 'desktop' folder from C:\Documents and settings\... or directly accessed it from the Desktop, or from My Computer\Shared Documents\.. and so on.
Example: Redundant paths for an XP machine
Desktop\My Computer\C:\Documents and Settings\ProfileName\My Documents\..
Desktop\My Documents\..
Desktop\ProfileName's Documents\..
Desktop\ProfileName\My Documents\..
Script
This script (download link below) parses the data from an image or an NTUSER.DAT file. It tries to remove all redundant paths (removing duplicates), althought sometimes you would want that (dates/times on the duplicate entries may be different).
Ideally, you want to run this against a full disk image (just because it tries to resolve ProfileName if not found in registry hive), but it will parse out individual NTUSER.DAT files also. It will process profile hives stored under System Volume Information (System Restore) also! It has been mostly tested on XP but has been tweaked to run on Vista and Windows 7.
Update: There appear to have been quite a few additions to the format with Windows Vista and 7. Also a new location for these entries:
C:\Users\ProfileName\AppData\Local\Microsoft\Windows\UsrClass.dat
The additional paths are noted in section above in "About the Bag format".
Update (May 26 2010): New version of script available.
Download here
