Access local drive with UNC and mklink on Windows

March 4, 2012

I set up a Tomcat app server on a ‘virtual’ hard drive. The virtual drive is assigned a local hard drive letter. But when a Groovlet attempts to access a local file, the file path used is on the default local drive, C:. How to allow access to the non-local file?

Symbolic links
There are probably many ways to do this and of course corresponding security concerns. One of these is to use Universal Naming Convention (UNC) paths. My quick approach is to just use file directory symbolic links. These are supported in most windows versions. On Windows 7 the MKLINK command is available.

Example
You want to access the folder w:\a\b\c on the virtual drive. On the “real” drive you create a link:

c:
mkdir a\b
mklink /D c w:\a\b\c

Now when the app server access the “c” folder via a UNC, \\mymachine\a\b\c, it will find it.

Duplicate post on other blog: Access local drive with UNC and mklink on Windows


Show hidden windows utility on PC

March 4, 2012

Sometimes a program may crash or show deranged behavior and one of the dialogues cannot be accessed. This just happened to me. A program looks like it is stuck in a thread issue. When I look on its thread list it seems to be waiting for user input. There is no dialog visible. Clicking on the program’s window just gives the waiting mouse pointer. Sure you can just kill the program but this may not help you find out what the problem really is.

I remember years ago I used a utility that could show these hidden windows. Good luck searching on web for something like “show hidden windows” or better phrases. You won’t find it. Well, yea, you could. I did. But, I’m good, smirk.

Note that this utility is pretty basic, just shows a list of window objects. Some of them should not be unhidden, they will lock the utility itself or cause other problems. Perhaps there is something out there that is better? Seems this should be part of the Sysinternals utilities.

Links

  1. Unhider
  2. Windows Sysinternals

Using Powershell instead of cygwin for scripting

February 27, 2012

In this blog entry, Windows Tricks For Whacking Lingering Java Processes …, the author resorts to the cygwin linux shell to automate an admin need.  Now cygwin is awesome, bringing some of the best tools from *nix.  However, with Powershell, Microsoft finally transformed the Windows command line into a powerful and usable administration resource.

Below a line from the bash script the author used to find each process given a process name and then loop thru each process ID and terminate it.   Note how the script requires a good understanding of bash scripting and in addition multiple Linux utilities  ps, grep, and sed.   The most important line in the script is:

found=` ps -aW | grep $procText |  sed -e's/^\s*\([0-9][0-9]*\).*/\1/' `
Here is how this works.
ps -aW:   show all processes and windows too.
grep $procText:   only use lines that contain the command line string
sed -e:   run stream editor inline
s///:   substitute
^\s*:  all beginning white space
\([0-9][0-9]*\):  find two or more digits, remember these.
.*:  any characters.
/\1/:  replace with the found digits

Very complex!!!! But, to *nix users this is nothing. Its even fun to do. With a Linux command line you can rule the world. Note that in the above script line, it’s just text processing.

Powershell has a different approach. In Powershell one works with objects (in the OOP sense). Thus, instead of transforming everything to text, one manipulates properties of objects. One pipes object instead of text.

To do:
Show how to do this in Powershell. Unfortunately, wrote the above a long time ago and lost what little Powershell mojo I was developing. Any help?

Update
3/18/12: The powershell approach will probably be expanded from something like this, which just lists the processes by name and process ID:

$strComputer = "."

$colItems = get-wmiobject -class "Win32_Process" -namespace "root\CIMV2" `
-computername $strComputer

foreach ($objItem in $colItems) {
      write-host "Name: " $objItem.Name
      write-host "Process ID: " $objItem.ProcessId
}


Hierarchical Storage Management, from drive to cloud

January 7, 2012

Years ago I came upon the HSM concept. Is this now applicable in today’s networked world for personal use? I propose HSM can be extended to encompass the Cloud.

HSM is an enterprise data storage technique, a tiered storage. Data is automatically moved from expensive but fast storage systems like hard disk arrays to cheaper but slower systems like optical or tape drive.

Conceptually, HSM is analogous to the cache found in most computer CPUs, where small amounts of expensive SRAM memory running at very high speeds is used to store frequently used data, but the least recently used data is evicted to the slower but much larger main DRAM memory when new data has to be loaded. — http://en.wikipedia.org/wiki/Hierarchical_storage_management

In the consumer world we have in our PCs fast hard drives and on mobile devices fast flash memory. A simple two tiered system would migrate least recently used files from local media to the cloud. In the home or SOHO environment, a three-tiered system is possible. Solid-State Drives (SSD) of modest size could be the 1st tier, SATA disk the 2nd, and finally, Cloud services can provide the 3rd tier.

For example, you have a PDF on your system that is a great resource, but you haven’t used it in a few weeks. The HSM manager would take that file and move it to the cloud (secure, private, encrypted, …., of course). In its place, to allow access by the user, is a link to the HSM managed storage location. Next time you use the file it will be migrated back to the local storage (but now also backed up in the cloud).

This is really an application of “file virtualization“.

Note that the HSM in enterprise systems is not simply based on “files” but on the underlying storage mumbo jumbo (frames, and all that).

This approach could make the potential future Windows 8 ‘Storage Spaces’ be even more useful. On *nix OS this is possible to implement now. It probably already is.

Demo
Here is a conceptual demo. We’ll use a known cloud storage service provider like Dropbox. As far as I know, Dropbox does not offer HSM.

On your PC you set a property on various folders that makes them eligible for HSM monitoring. This could be accomplished using a GUI and drag&drop. The HSM will immediately copy the folders to the SSD on your system or the main hard drive, if the files are not already on the fastest subsystem. In the original location of the folders, a link to the new locations will be created (soft links?). The end user will not see any difference. Kind of like “web folders” or WebDAV protocol.

After a period of time, the local HSM monitor will record which files have not been used and invoke the Dropbox local service to stream the files to the cloud. All that remains on the file system are links to the remote files; storage space is reclaimed.

Updates
Feb 26, 2012: Another company will be competing with DropBox. As above it allows the user to designate specific folders to participate in cloud storage. Since remote files will be slower to access, this company will attempt to “predict” which files would be used more often. See this article.

Further Reading


Bob Dylan “With god on our side”


Windows 8 Storage Spaces?

January 7, 2012

Finally Windows may get a file system that adds features that have been around for years in Linux and Unix.

The state of the art in file system design could arguably be ZFS which is used in Solaris and a few other places.

Storage systems seem like very complex engineering systems. I wonder if Microsoft would have just been better off not reinventing the wheel and just licensing ZFS technology or something else? Just search for user horror stories regarding any of the storage improvements on Windows, from drive extender, encryption, WHS, etc.

And what ever happened to the object file system, the DB based file system?

Storage Spaces is somewhat like ZFS, although it has no deduplication and lacks other ZFS features. However, it is a start – and Microsoft will probably add features such as snapshots, replication, deduplication, and, maybe, compression. El Reg also thinks that there could be a Hyper-V virtualisation angle to this – and more is to come. — http://www.theregister.co.uk/2012/01/07/windows_8_storage_spaces/page2.html

Update
Jan 17, 2012: Ah, so Microsoft is working Resilient File System (ReFS).

Microsoft mentioned all the important keywords when promoting ReFS with words like resilience, availability and the use of storage pools, something made popular by Sun’s – and now Oracle’s – ZFS.
Source: The Inquirer (http://s.tt/15gRk)

Further Reading


VOWS “Winter’s Grave” OFFICIAL video


Debugging: copying .git folder hangs Windows Explorer

September 26, 2011

Windows Explorer started hanging when using copy & paste of folders. I noticed that it only involved folders that had a “.git” subfolder.

Symptoms

  • Copy a folder and explorer does not refresh; you have to hit F5.
  • After the manual refresh, click on a folder and the explorer gets an hour glass.
  • The explorer.exe CPU usage reaches 50% and higher.
  • Only happens when a subfolder was named “.git”.
  • .git folder could even be empty.

Easy approach unsuccessful
I uninstalled Git and TortoiseGit then restarted the system. Still happening. Hmmm, that should have removed everything. Nope, I looked in the Registry and TortiseGit was still hanging around. I removed all traces and then even used ShellExView tool and could not find any shell extension that could be doing this. Using that tool I disabled all non-Windows shell extensions. Still happening.

More tools to help
This happened a month ago so I don’t remember all the steps I took. I used some of the SysInternals tools but they didn’t help much; you need a lot of internal Windows details to really use some of them.

However, I think it was using the Process Manager and looking only at file activity that all the info finally gave me a clue.

Progress

5:23:14.4666723 PM	explorer.exe	5096	ReadFile	C:\Documents and Settings\t16205\Start Menu\Local Disk (C)\target.lnk	SUCCESS	Offset: 0, Length: 299
5:23:14.4668257 PM	explorer.exe	5096	QueryInformationVolume	C:\Documents and Settings\t16205\Start Menu\Local Disk (C)\target.lnk	SUCCESS	VolumeCreationTime: 9/15/2009 8:47:39 AM, VolumeSerialNumber: 64B6-6315, SupportsObjects: True, VolumeLabel:

It seemed to be hanging at that point!

Solution
I looked in the Windows Start Menu list and saw that I had placed a link of the C: drive there. Not a junction or anything fancy, just a drag and drop of a shortcut (I think). I removed that link and the problem was solved.

Back to Git
Reinstalled Git and all is fine, knock on wood.

No explanation yet
Not really debugging, just floundering with a purpose. But, why would this only effect folders named “.git”, when even Git is not even installed. Strange.

OS
Windows XP Professional

Git
TortoiseGit
mysysGit

Tools
WhatIsHang v 1.10:
ShellExView v 1.66
Process Explorer v15.04
Process Monitor v 2.96
brain


VB file sharing with Ubuntu guest and Windows 7 host

January 15, 2011

VirtualBox 4.0 can now mount shared folders automatically!

Yes, you could always do it with a few edits of some system config files, but this puts it at a non-geek average user level, a file sharing for idiots perhaps. The instructions on creating shares is in the VB manual section 4.3.2 Automatic Mounting.

I tried it and can now see my full C drive on Windows 7 from the Ubuntu guest running in VirtualBox (VB). But, at first, it did not work, and a previous manually mounted share also stopped working, both with access permission problem. The big detail I missed:

Access to auto-mounted shared folders is only granted to the user group vboxsf, which is created by the VirtualBox Guest Additions installer. Hence guest users have to be member of that group to have read/write access or to have read-only access in case the folder is not mapped writable.

So after you create the share you have to add users to the vboxsf group. Using the Ubuntu UI you navigate to System, Administration, Users and Groups, then Manage Groups. Now find vboxsf and show its properties, from there you can add users. Reboot.

From the Oracle VM VirtualBox manual:

4.3.2. Automatic mounting
Starting with version 4.0, VirtualBox can mount shared folders automatically, at your option. If automatic mounting is enabled for a specific shared folder, the Guest Additions will automatically mount that folder as soon as a user logs into the guest OS. The details depend on the guest OS type:

With Windows guests, any auto-mounted shared folder will receive its own drive letter (e.g. E:) depending on the free drive letters remaining in the guest.

If there no free drive letters left, auto-mounting will fail; as a result, the number of auto-mounted shared folders is typically limited to 22 or less with Windows guests.

With Linux guests, auto-mounted shared folders are mounted into the /media directory, along with the prefix sf_. For example, the shared folder myfiles would be mounted to /media/sf_myfiles on Linux and /mnt/sf_myfiles on Solaris.

The guest property /VirtualBox/GuestAdd/SharedFolders/MountPrefix determines the prefix that is used. Change that guest property to a value other than “sf” to change that prefix; see Section 4.6, “Guest properties” for details.

Note
Access to auto-mounted shared folders is only granted to the user group vboxsf, which is created by the VirtualBox Guest Additions installer. Hence guest users have to be member of that group to have read/write access or to have read-only access in case the folder is not mapped writable.

To change the mount directory to something other than /media, you can set the guest property /VirtualBox/GuestAdd/SharedFolders/MountDir.

Solaris guests behave like Linux guests except that /mnt is used as the default mount directory instead of /media.

To have any changes to auto-mounted shared folders applied while a VM is running, the guest OS needs to be rebooted. (This applies only to auto-mounted shared folders, not the ones which are mounted manually.)

System
VirtualBox: 4.0.0 r69151
Ubuntu: 10.10 (maverick)
Kernal Linux: 2.6.35-24-generic
Window 7:
PC: p6230

Further reading

  1. Dupe post: VB file sharing with Ubuntu guest and Windows 7 host
  2. VirtualBox Shared Folders in Ubuntu 10.10 Guest with Windows 7 Host
  3. VirtualBox

 


Ratkje- Vox


Follow

Get every new post delivered to your Inbox.