Windows hooks & call sequence

I ran into an interesting little issue at work the other day. There is this program that I am working on which happens to embed Office documents into a browser control hosted in a CHtmlView derived view window (that's right, we use MFC). The requirement was to popup a little toolbar with a save button whenever the user did some in-place editing on whatever's displayed in the browser control. We already knew how to detect whether at a given point in time the document currently displayed in the browser has been edited. This was done in this manner (error checks omitted for brevity):

bool IsDirty()
{
    //
    // get hold of the html view object somehow
    //
    CHtmlView *pView = GetViewSomehow();

    //
    // get a pointer to the HTML document object
    //
    CComPtr<IDispatch> spDocument( pView->GetHtmlDocument() );

    //
    // turns out that you can QI on the doc object to
    // get a "IPersistStorage" object which is directly linked
    // to whatever happens to be embedded in the browser
    //
    CComPtr<IPersistStorage> spStorage;
    HRESULT hr = spDocument->QueryInterface( IID_IPersistStorage, (void **)&spStorage );

    //
    // now we just call "IPersistStorage::IsDirty" to figure
    // out whether the doc's been edited
    //
    hr = spStorage->IsDirty();
    return ( hr == S_OK );
}

The question of course was to figure out when this routine will be invoked so we can display the toolbar. We first came up with an approach that involved continuously polling for changes by making some creative use of CWinApp::OnIdle. But that resulted in short bursts of CPU usage spikes and while the solution worked, it somehow didn't feel right!

The next thing we tried was to see if Windows hooks can be put to some use here. We quickly set up a system-wide keyboard and mouse hook which "phoned home" so to speak whenever an event occurred by posting a custom message to a window. Whenever this message is received by the window it would call our clever little IsDirty routine to check if the document has been modified and get the toolbar displayed if need be. I let out a satisfied little burp at this point and compiled, linked and hit Ctrl+F5.

I loaded up a PowerPoint file into the browser control and pressed a few keys and whopeeee(!) the toolbar appeared straightaway. On doing some additional testing however I discovered that the damn thing showed up only with the second key press and not immediately after the first one! As it turns out, Windows delivers messages to hooks before they are delivered to the target application. So my hook was getting the keyboard event before PowerPoint was getting it and the call to IsDirty was consequently returning false as PowerPoint hadn't had a chance to mark the file as having been modified yet.

This thing drove us a little nuts until of course we figured a way out. I even made a newsgroup post on this issue (with no response by the way). The solution in the end turned out to be quite simple.

Well-behaving hooks are required to call the CallNextHookEx function before returning from the hook routine. This is to let other hooks that are installed on the system have a go at the message. I had done a PostMessage to the application window before calling CallNextHookEx like so:

PostMessage( hwndNotify, WM_HOOK_NOTIFY_MOUSE, 0, 0 );
return CallNextHookEx( NULL, nCode, wParam, lParam );

I made a small change to the order of invocation in this manner:

LRESULT lResult = CallNextHookEx( NULL, nCode, wParam, lParam );
PostMessage( hwndNotify, WM_HOOK_NOTIFY_MOUSE, 0, 0 );
return lResult;

And voila! it started working! It must be pretty evident what the issue was by just looking at the change that was done. Looks like CallNextHookEx, apart from calling other hooks that may have been installed also actually delivers the message to the target application before returning. In this case, this was just what the doctor ordered :)!

Cool eh?!

comments powered by Disqus