Simple Managed Inter Process Communication(IPC) Framework

If the need is just to do a cross process communication within the box, it'd be bit heavy to use frameworks such as WCF as it comes with a learning curve and complex setup. However  home grown solutions such as messaged based implementation suffer from flexibility and heavy maintenance.

 The simpleIPC framework tries to strike right balance with the interface based programming coupled with zero setup.

IPC Overview

A lightweight Inter Process Communication (IPC) across process within the same PC in .Net can be implemented using named event object or Window kernel object and shared memory.

• Interface based programming
• Duplex communication support
• Servers are identified by a unique string to which clients can generate a proxy to communicate. Some of the features:
• Servers/ Clients generate Stubs/ Proxy based on an interface for communication using reflection.
• As the proxy is based on the RealProxy object, intellisense is also supported in the Visual Studio IDE editor. Easy to debug, just a single file of code.
• To access a managed server from an unmanaged client, reversePinvoke can be used.

As shown in  the diagram above. IPC consists of a Client, Proxy, Stub and Server as described below

• Client uses proxy based on RealProxy to make API call
• Proxy serializes the input data using binary formatter to shared memory and signals stub
• Stub deserializes the input data using binary formatter from shared memory and makes the call on the server using reflection
• Stub serializes the results to shared memory and signals Proxy
• Proxy deserializes the results from shared memory and returns it to the client

Implementation

Comes in two flavors windows kernel object based or named event kernel object based.

Windows based uses a window kernel object for implementing the server

As shown above multiple Windows are hosted in a thread. Each thread hosts a server. 

• A Window based container can supports multiple servers per thread
• A window based server cannot work across window stations

Named object based uses a named event object for implementing the server.

• A Named event object based container supports only one server
• A named event object based server can work across window stations

Example

The interface  ICallInterface shown below is implemented by the server and ICallbackInterface is implemented by the client.

namespace Example
{
    [Serializable]
    public class regdata
    {
        public string name;
        public string regid;
    }

    public interface ICallInterface
    {
        string current { get; }
        string register(string name, string cbservername, out int ticket);
    }

    public interface ICallbackInterface
    {
        void update(int ticket, regdata data);
    }

}

Windows Simple IPC  Demo

Server Process

Simple IPC Server name for ICallback : winserver

Client Process

Simple IPC server name for ICallbackInterface : winclient

Named Object Demo

Server Process

Simple IPC Server name for ICallback : namederver

Client Process

Simple IPC server name for ICallbackInterface :  namedclient

Operation

Server process starts

client process starts

Client:

creates proxy for server

calls register() on server using proxy

Server:

executes method register() 

server returns success with Ticket #100

Client:

results are received from proxy : success, Ticket:100

Server:

creates proxy for client implementing callback interface

calls update() method on the proxy

. 

Client:

executes update() method

Source and Binaries can be found here.

Unattended System monitoring with ProcessMonitor and UI automation

ProcessMonitor from Microsoft is hugely popular and widely used to monitor events such as registry or file related updates. 

If Process monitor is used to track an event  (e.g., a registry change) system wide, in an unattended scenario, some customization will be needed since ProcessMonitor  creates a large amount of log files in a short span of time i.e., around 5gb in 30 minutes. 

The plan is to check Process monitor log files every 30 minutes for the event, take a snapshot and clear the logs. Using a task scheduler task and UIA this can be accomplished as discussed below.

Background

It was noticed that  on a particular windows 10 machine, the windows scaling factor changed from 100% to 150% randomly over a period of 1 week. The plan was to track changes made to the registry entry hkcu\control panel\desktop\logpixels.

Implementation

The solution should be generic so that it can be deployed seamlessly on different PCs.

When done manually, following steps are executed.

Steps

1. First time, folders for the pm logs and captured events are created. This is done by importing  the configurations from ProcMonConfiguration.pmc.
2. After that, when a specific user logs in, the unprocessed  logs are scanned for the event and cleared
3. Every 30 mins logs are scanned for the event, logs are saved if found and then logs are cleared

Step 1

 Init.cmd will do the initial setup.

Add a filter  for registry entry as shown below:

Change file backing as shown below.  A dedicated folder in required.

A scheduler task startpm is created that runs on logon of specific user to  run startpm.cmd described below.

A scheduler task savepmlog  is created that runs startpm.cmd for a specific user every 30 minutes to check for the event , take a screenshot and  clear the logs.

Step 2

startpm task is executed on logon of a specific user. This tasks runs savepmlog.cmd.

Step 3

savepmlog  task is executed every 30 mins after logon of a specific user. This tasks runs savepmlog.cmd.

Init.cmd

This script will do initial setup as discussed above.

       
@echo on
rd /s /q c:\temp\procmon
md c:\temp\procmon\capture
md c:\temp\procmon\pmlogs
schtasks /create  /f /sc onlogon /tn startpm /it /ru rvvya /tr "%~dp0\savepmlog.cmd"
schtasks /create  /f /sc minute /mo 30  /tn savepmlog /it /ru rvvya /tr "%~dp0\savepmlog.cmd"
start "" "%~dp0\..\bin\Procmon64.exe" /accepteula /terminate
start "" "%~dp0\..\bin\Procmon64.exe" /quiet /loadconfig "%~dp0\ProcmonConfiguration.pmc"
       
 

savepm.cmd

This script will check logs for the event and saves it along with a screenshot if found and then clears all logs.

       
@echo on
setlocal enabledelayedexpansion
del C:\temp\procmon\capture\snapshot.bmp>nul
del C:\temp\procmon\capture\logfile.csv>nul

tasklist /fi "imagename eq Procmon64.exe" | find /i "Procmon64.exe">nul
if !errorlevel! equ 0 start "" /wait "%~dp0\..\bin\Procmon64.exe" /Terminate
start "" "%~dp0\..\bin\Procmon64.exe" /quiet /openlog C:\temp\procmon\pmlogs\uiatest.PML
echo saving....
start "" /min /wait "%~dp0\..\bin\savelog.exe"
call :rename_logfile
goto :eof

:rename_logfile
set logfile=logfile_%date%_%time%
set logfile=%logfile::=_%
set logfile=%logfile:/=_%
find /c /v ""  C:\temp\procmon\capture\logfile.CSV | find /i ".CSV: 1">nul
if !errorlevel! equ 0 (
del C:\temp\procmon\capture\snapshot.bmp>nul
del C:\temp\procmon\capture\logfile.csv>nul
echo %date% %time% not found >> C:\temp\procmon\capture\results.log
) else (
move C:\temp\procmon\capture\snapshot.bmp "C:\temp\procmon\capture\%logfile%".bmp
move C:\temp\procmon\capture\logfile.csv "C:\temp\procmon\capture\%logfile%".csv
echo %date% %time% found >> C:\temp\procmon\capture\results.log
)
exit /b
      
 

Savelog.exe

This is driven by UIA. This executable is launched by savepmlog.cmd. It takes a snapshot, saves pmlog files and then clears them. 

Deployment

The solution is deployed as shown below:

\bin

ui automation executables and the libraries and processminitor executable.

\scripts

contains task scheduler scripts described below and setup script

Demo

The following movie shows actual operation

1. Regedit is started to add an event

2. savepmlog task is triggered to check and record.

Output 

The  output files are available in C:\temp\procmon\capture folder.

snapshot (logfile_11-06-2022_15_26_36.34.bmp)

saved log file (logfile_11-06-2022_15_26_36.34.csv)

"Time of Day","Process Name","PID","Operation","Path","Result","Detail"

"15:26:06.7849475","regedit.exe","10344","RegQueryValue","HKCU\Control Panel\Desktop\LogPixels","SUCCESS","Type: REG_DWORD, Length: 4, Data: 150"

results.log

11-06-2022 14:32:30.01 not found 

11-06-2022 15:00:22.52 not found 

11-06-2022 15:26:36.36 found 

Source and Binaries can be found here.

User Interface Automation Framework

The Microsoft UI Automation APIs enable navigation of user interfaces programmatically.

Some of the areas of application are UI Automation testing,  manufacturing Quality checks, Accessibility,

The UIAFramework  is a light weight framework that can be used to achieve UI Automation.either by Scripting languages or C#. It has two components ControlDBTool and UIADriver. The UIAFramework is based on .net CLR 3.5 and works with Windows XP and above. The process involves two steps.

1. Identify controls participating in UI automation in an user interface and generate a control database using ControlDBTool.
2. Using the UIADriver component, manipulate the control by invoking a method  or property from the supported control patterns of the control. For e.g., Invoke method of Button control.

ControlDBTool

ControlDBTool is a GUI based tool that aids identification of UI elements that participates in UI automation. The users can basically select UI elements from an application and save them into a file based repository. ControlDBTool also supports record and play feature where code can be generated from user actions.

UIADriver

UIADriver is an easy to use managed class library that provides an interface to simulate user interactions on various UI elements such as Buttons, Grids, combo boxes etc. This is exposed to script based and unmanaged clients via COM.
UIADriver supports both UIA (user interface automation) and MSAA (Microsoft Accessibility Api) technologies. These basically provide the foundation for UI automation and are part of the OS. UIA is the successor of MSAA and can be used for both WPF and Win32/Winform based UI controls.  However, it exhibits anomalies with certain winform based controls. In such cases, MSAA can be used to fill the gap.
The code generator of the ControlDBTool discussed earlier uses the UIADriver library to perform UI operations.

Example

In this example, calucalator.exe is launched and  multiplication calculations are performed as discussed below.

1. Launch calucalator 

2. Launch ControlDBtool and set toolbar only mode

.

3. Click on record button and perform following actions on the calculator using mouse

4. click digits 8 7 8 4 x 1 6 x 1 0 4 0 as shown in the below

5. stop recording the controldb tool 

6. save the recording

7. recorded actions are listed as below. if there is any unmapped event, it needs to be mapped to the action.

8. set  controldb tool to regular mode

9. save the recording

10. click on generate code and save the selection

11. select all the actions

12. select C# code from the dropdown

13. Save the code to calc.cs

14. edit calc.cs file and add highlighted code in main() and save.

	   
public static void Main()
{
	string brs = null;
	System.Diagnostics.Process.Start("calc.exe");
	System.Threading.Thread.Sleep(2000);

	UIAAutomationElement.UIADriver.SetAutomationElement (objsample.button_Clear_clearButton_1294_432);15. open a command window from the saved location

16. compile the code to generate exe using following command

17. execute runcalc.exe. the playback looks as in the video below

Source and Binaries can be found here.

File Splitter and Merger Tool for Transmission

What's File Splitter and Merger Tool?

It's an command line utility to truncate a big file into small pieces, later to be assembled.
Useful for sending big attachment files in mails, copying a big file into floppies etc.

How does it work?

During Disassemble, input for the source file, destination folder and truncation size are accepted. Later the file is split into multiple files with the name filrname.nn where n is the file number in the destination folder.  e.g., Test.data.00,  Test.data.01 etc

During Assemble, input for the destination file that will be assembled, source folder that contains truncated files are  accepted. Later the destination file that will be assembled back from the multiple  truncated files from source folder .

How to use it?
let's say if we want to break a big file d:\data\data.mdb of 10MB into eleven(11) 950K files:
open a Command window and type the command below.

FileSplitterMerger D d:\data\data.mdb  950000

you can see that 11 files d:\data\data.mdb.00 ..... d:\data\data.mdb.10  are created.

To assemble back, open a Command window and type 
FileSplitterMerger A  d:\data\data.mdb  
you can see the d:\data\data.mdb created after the merge

Source and Binaries can be found here.

Script to Zip and Unzip Files using Window shell

Often times applications require files to be zipped to a .zip file and extract files from the same. makezip_unzip.vbs  uses Windows shell do this easily.

Create Zip File Example

1. Arrange all folders, sub folders and files matching zip file structure inside a "stage" folder. Example, d:\temp\reports
2. Execute  in command window

           makezip_unzip.vbs   "d:\temp\reports"   "d:\reports.zip"

if not exist, d:\reports.zip will be created.

Extract from Zip file Example

Execute in command window

           makezip_unzip.vbs    "d:\reports.zip"   "d:\temp\reports"

If not exist, d:\temp\reports folder will be created to receive files from the .zip file. 

Note that full path must be supplied to the arguments.

Source and Binaries can be found here.

Innovative File organizer with Duplicates finder and Hierarchical viewer

No matter how big the hard drive's capacity gets, the prevalent issue of lack of storage remains.

When there is an urgent need to free up space, getting a fair idea of  the storage distribution becomes paramount.

FileOrganiser tool attempts to fill the gap by providing a hierarchical  view of the storage distribution along with options to sort data in multiple ways.

File organizer with Duplicate file finder has following features.

Folder View

Lists folders and files in a single screen hierarchal order along with their disk sizes and file count.

• The files can be viewed hierarchically  and can be sorted by File count, File Size and File Names.
• The file size can be viewed in different disk size units such as KB, MB and GB
• Selected Files can be deleted or path can be exported to file or copied to another folder.
• Files can be selected individually or entire folder can be selected using toggle selection
• Width of file display can be changed

Duplicates View

Multiple copies of the same file such as pictures gets spread across the disk. It also becomes important to keep track from security Point of View.

The user interface is same as compare folder view discussed below except duplicates are searched in the same folder.

Compare Folders View

Compares two different folders and lists duplicate files in hierarchical order  along with individual files.

Duplicate Hierarchical View

Duplicate Flat View

Non Duplicate Hierarchical view

• The duplicates files can be viewed hierarchically  or flat and can be sorted by File count, File Size and File Names.
• The file size can be viewed in different disk size units such as KB, MB and GB
• Selected Files can be deleted or path can be exported to file or copied to another folder.
• Files can be selected individually or entire folder can be selected using toggle selection
• Width of file display can be changed

Export

Exports files along with their unique MD2 values. Useful for large directories where faster processing can be done using SQL based queries.

Individual entries are separated by |

Source and Binaries can be found here.

A Backup, Restore, and Update Tool with a refined approach

Let's face it, over the period number of files grow, change, removed, copied and moved etc. Keeping track of these will be a challenge unless they are backed up and tracked to restore when needed.

The Backup and Restore Tool discussed here has following features:

• Create Backup collections and add / remove folders that will be backed up.
• Add exclude folder list such as system folders. e.g., c:\window
• Track files by CRC so that only a single copy of that file is stored.
• Files are backed up to a password protected archive.
• Each archive's metadata is also password protected.
• During backup, an integrated view is presented where each folder is indicated as archived and partial , modified or changed.
• During backup, an integrated view is presented where each file is indicated as archived and modified 
• During restore, an integrated view is presented where each folder is indicated as same, deleted and changed 
• During restore, an integrated view is presented where each file is indicated as same, deleted and changed 

Main

The following diagram illustrates the backup and Restore tool.

New

Creates new archive. The GUI is shown below.

Operation

1. Enter a name for the archive. e.g., TechBlog
2. select a location  for the archive. A new folder will be created under this folder to store contents of the archive. c:\archive\TechBlog.
3. Enter a description for the archive
4. Select folders to backup. e.g., d:\techblog. The checkbox of the folder to be backed up should be checked to take the backup.
5. Folders that needs to be excluded from backup such as system folder can be specified in exclude.txt under the collection folder.
6. After Clicking OK, user is asked to enter a password for the archive. This password is used for storing metadata of the archive.

Edit

Once the archive is created, edit can be used to add additional folders to backup or select deselect folders for taking backup.

Backup Tool

A collection is a set of folders that needs to be archived in multiple backups. Each backup contains a snapshot of the archived folders at the time backup was taken. 

Operation

1. First select folders to backup as discussed above and Click Backup button
2. An integrated view is presented to select files/folders and status of the file and folder items. 

The status of a file item can be as below.

• New - New file. Never backed up.
• Changed - Existing file changed after last backup. 
• Archived - Existing file that didn't change since last backup.

Similarly the status of a folder item can be as below.

• New New Folder. Never backed up.
• Partial Existing folder structure changed since last backup. 
• Archived  No changes since last backup
• 
  

3. Finally, Check the checkboxes to choose the files that needs to be backed up. Click Backup Now button to start taking backups. The backups will be saved into a internally generated password protected 7zip file. 

 Restore Tool

The restore tool can be used to restore files from previously archived backup.

Operation

Select a backup from the dropdown list to restore.

Files are synched with disk if "sync with folders" is checked. This will compare the crc of the existing files with the previous backups and determine which files needed to be restored. 

Each file can have three states -  Same, Changed or Deleted.

• Same means the file has not changed hence no restoration is needed.
• Changed means the file has changed hence restoration is needed
• Deleted means the file is deleted  hence restoration is needed

In the same way, A folder can have one of the three states -  Same, Changed or Deleted.

• Same means the folder has not changed.
• Changed means one or more files in the folder has changed hence restoration is needed.
• Deleted means the entire folder is deleted  hence restoration is needed.

By default all the files from the previous backups are listed. If  changes only is checked, only files from the selected backup are listed.

If Hide Same is checked, all the unchanged items are removed from the display.

Export exports file  items listed to a csv file along with the crc.

Refresh Resyncs with disk for latest changes.

Finally, Check the checkboxes to choose the files that needs to be restored. Click Restore button to restore. As shown below, the files can be saved to a different folder or to the same original folder where they existed.

 Update Tool

It's a hybrid of Backup Tool and Restore Tool. The Update tool can be used to update previously archived backup. Folders and Files can be added and removed from the archive without having to use Backup Tool. This is helpful incase the original backup is not available.

Operation

Select a backup

Expand the tree and select the folder that receives files

Drag and drop files. after acknowledgement  files will be added automatically.

Source and Binaries can be found here.

Desktop Virtualization of Released Software using Virtual Hard Disk

Software Applications in regulated domains  use customized  OS based  on an embedded OS such as WES7 or Win10 IOT. This is for restricting access and  also to provide a highly controlled environment.

The application software installation can be time consuming as it involves first formatting and installing the OS; then installing application software. With multiple releases of the software and limited resources, installation of an older release can be painful especially if it's DVD based due to wear and tear.

A Virtual Hard Disk or  VHD is nothing but a flat file  on the disk of host PC, which can be mounted as a Hard disk on the system with multiple partitions. Further a VHD can be made bootable by adding an entry to the boot menu. A VHD can be created using DiskMgmt control panel app or using diskpart.exe.

A VHD can come very handy where OS + Application can be pre-installed on a VHD. It simply can be mounted and made bootable on demand. In other words, a single PC can run multiple installations on demand without requiring an installation. The file can be even copied and deployed on a different but compatible PC.

Example

Consider a typical software application that runs on a customized embedded OS. The volume layout can be as below.
C:  OS + Application Software
D:  Data

 

Create a VHD

A VHD can be created using DiskMgmt control panel app or diskpart.exe. The example below creates a VHD with a matching volume layout as shown above.
After running the script, the VHD can be visualized as shown above  diskmgmt.msc

Load  OS into VHD

The next step is to format the active partition on the VHD and load OS into it using imagex command
Note that H: is the system partition as shown above.  This usually loads syspreped image of the OS
imagex /apply Installer.wim 1 H:

Make VHD bootable and make it default

bcdboot h:\windows
Boot files successfully created.
bcdedit /set {default} description "Some App 1.0"
The operation completed successfully.

Reboot

After reboot a new boot menu entry is added as shown below and is automatically selected to boot from.

Load Application Software

Once OS is installed, the Application software will be loaded as a part of the installation.

Source and Binaries can be found here.

Using bootable Virtual Hard Drive as Media instead of USB Drive/DVD Disk

Installing commercial software having embedded OS  such as WES7 or Win10IOT, uses various types of media such as CD/DVD or USB. This is mainly because of deployment of custom Operating system. These type of installations cannot be automated because of  need of manual operations such as boot options, bios password etc.

A bootable Virtual Hard Disk( VHD) provides a perfect solution. Following describes the process in detail.
1. Duplicate the USB disk image into a Virtual Hard disk. For example, The VHD contains two partitions. The first is bootable partition containing WinPE OS. The second is Service Disk containing license files.

2. Load the OS

The OS Image in an USB has following structure:

The sources folder contains the boot.wim which is loaded by the OS. However for the VHD, the entire boot.wim should be extracted into the bootable drive.

In addition, customization of the scripting may be required for correct deployment. It can be done starting with modifying windows\system32\startnet.cmd file. This script is first that is executed.

3.Adding boot entry and make it default.
The last step is to add an boot entry to the VHD file. After adding the entry system is rebooted. The system loads bootable VHD drive and loads the WinPE OS the same way in case of an USB drive.
bcdboot u:\windows
Boot files successfully created.
bcdedit /set {default} description "Some App 1.0"
The operation completed successfully.

4.Reboot the system

After the system restarts, the system will automatically select the bootable vhd to start.

Source and Binaries can be found here.