Tools: MacApp
Advanced Search
Apple Developer Connection
Member Login Log In | Not a Member? Contact ADC

TidyHeap Release 2

TidyHeap Release Notes:
Last Revision: 11.15.96
Copyright © 1985-96 by Apple Computer, Inc. All rights reserved.

Table of contents:


Description:

TidyHeap is tool for debugging and fine tuning C++ applications. It is a C++ library that can be built as a static library and included in your project. It can also be compiled directly into your project. TidyHeap tracks blocks you create with new or delete and notifies you if these blocks aren't deleted. It can also verify the integrity of all or a single block in your heap.

TidyHeap was designed with extensibility in mind. You can create 'Action' classes that operate on blocks of memory allocated by new or delete. TidyHeap ships with a few action classes in UTidyHeapActions.h , including a double delete watcher, a block verifyer, a incremental new failer, break on delete, and others.

TidyHeap operates by intercepting all calls to the global "new" and "delete" operators. During a "new", TidyHeap increases the memory size of the block requested and puts a header and trailer on the block of memory, it then creates a 'TTidyHeapBlock' and puts the TTidyHeapBlock into a linked list, and then stores a pointer to the TTidyHeapBlock in the header of the original block of memory. During a call to "delete" TidyHeap locates the header information from the incoming block pointer so that the proper block of memory and TTidyHeapBlock object is deleted correctly.

The TTidyHeapBlock object keeps track of the state of the clients block of memory and provides some helpful functionality such as verification, an ID, and some information about the block. The pointer to the TTidyHeapBlock can be retreived at any time from the address of the original block of memory. Because TTidyHeapBlock is an object, clients may subclass it to provide their own functionality.

TidyHeap has a extensible logging mechanism. A mechanism for logging to a text file is provide and installed by default. This file will be named 'MyApplication.log'.

TidyHeap is a "Singleton" class that is created automatically the first time it is accessed, usually during the first "new" of the client program. You may access the instance of TTidyHeap through the static method "Instance" in this way: TTidyHeap::Instance()->DoSomething(). If TidyHeap is not created at the time of the call, "Instance()" will create it and return a valid pointer to you. We recommend that new is not called during static creation time, or before the toolbox is initialized, although it will work.


FAQ:

  1. What are some good ways to track down my undeleted blocks?
    • Use NewDropper on your source. When the undeleted blocks are written to the logfile you'll see the file and line number of where the block was created.
    • Find the Id of the block(s) that is leaking and set a conditional break point with your debugger of choice in TTidyHeap::AllocateBlock. When the program reaches the break point you can look at the stack crawl and see where the block is being created. Note: you can get the block's Id programitically or by looking at the id of the block in the logfile.
    • Find the Id of the block(s) that is leaking and create a ' TActionBreakOnNew' object and run your Debugger of choice. When this block is created, the action will break into MacsBug and you can check out the stack crawl.
      Example: 
      TTidyHeap::Instance()->AquireNewAction( new TActionBreakOnNew(953) );
  2. What if I have blocks which I don't want to delete and don't want to be reported as leaking?
      You can exempt a block by installing a 'TActionExemptBlocks' or calling 'Exempt' on the block directly.
  3. I'm really irritated by the undeleted block message popping up everytime I quit my application, is there a way to turn it off?
      But of course! Open your applications settings file (titled something like 'myapp.prf') with TidyHeap Director and turn off that warning. TidyHeap will continue to function, it just won't irritate you with that message any longer.;-)

 


Requirements:

TidyHeap is PPC only (because of the dependency on 'TidyHeapSharedLib') and has only been tested with Metrowerks Codewarrior IDE. Theoretically, there isn't any reason why it shouldn't work on other compilers, it just hasn't been tested yet.

To link correctly, your application need the following:

  1. TidyHeapSharedLib must be included in the application project, and TidyHeapSharedLib must be in your applications directory or in the Extensions Folder.
  2. All TidyHeap source and header files must be included in the application project, either directly or in a library (which can be built with accompaning project).
  3. The procedure 'CreateTidyHeap' must be defined and included in your project.
  4. Make sure qTidyHeap is defined before include 'UTidyHeap.h' in your project.

    Check out this for more info.


Logging (Logf):

TTidyHeap provides logging functionality in a function called Logf. It works in the same way as printf. The output goes either to DebugStr, a logfile, or a user defined TTidyHeapLogger class. The reason this is part of TidyHeap is to provide an exetensible mechanism for clients to get the logging information that is independent of existing tools such as Metrowerk's SIOUX or even simple MacsBug.

For example;

TTidyHeap::Instance()->Logf("I have %ld blocks allocated.", numBlocks);

The way logf works is that it gets all the data off the stack and builds up a complete string, and then calls the currently installed TTidyHeapLogger object's virtual log method. The user can provide their own TTidyHeapLogger object to perform any function desired by subclassing TTidyHeapLogger and installing the object into tidyheap by calling TTidyHeap::Instance()->AquireLogger(). Note that sending NULL into AquireLogger effectively turns off logging.

Two TTidyHeapLogger objects are currently provided:

1) TTidyHeapDebugStrLogger: all output goes to DebugStr.

To activate this, call TTidyHeap::Instance()->SetToDebugStrLogger( doHalt ) . Note: "doHalt" is a boolean which if it is set to false TidyHeap appends a ";g" to all logged strings so that MacsBug does not stop excecution of your program.

2) TTidyHeapLogFileLogger: all output goes to a text file that is created by TTidyHeap. Note: this is the default logger. The file name of the log file is in this format: "Application Name.prf", and it is saved in the default location.

To activate this logger, call TTidyHeap::Instance()-> SetToLogFileLogger ()


Warnings (Warnf):

TidyHeap provides a warning mechanism that is independent of the logging functionality. This warning is output to both DebugStr and to Logf. You can enable/disable warnings by calling SetWantsWarnings with a boolean value. This defaults to true.

For example:

Example:

TTidyHeap::Instance()->Warnf("Uh oh, an error! (%d)", myErr);

Example:

TTidyHeap::Instance()->SetWantsWarnings(false);

Note: TidyHeap will warn you if you have undeleted blocks at termination time.


TTidyHeapBlock:

#include "UTidyHeapBlock.h"

TTidyHeapBlock is a class that holds important and useful data about the block of memory it is attatched to. The user block's header has a pointer to an instance of a TTidyHeapBlock (see header/trailer format below), i.e. there is one TTidyHeapBlock for every allocated block of memory.

Every block is uniquely numbered - this is the blocks' ID.

TTidyHeapBlock keeps track of its state by the member object TTidyHeapBitFlags, which uses a long for a collection of 32 flags. TidyHeap uses the lower word, but you may use the upper word for any purpose you like. Iif you flip any bits in the upper user word, be careful to preserve the value of the lower word.

Note: two interesting flags you may want to know about are kDelete and kDontDelete. kDelete is always set when the block is deleted. However, if the kDontDelete bit is set, the block won't really be deleted. This is how the CDoubleDeleteWatcherAction works (see below).

You can install a Delete Action into any block by calling myBlock->AquireDeleteAction( myAction ). This action will be excuted when the block is deleted.

You can verify the integrity of a block by called myBlock->Verify(). This method will return an error if the block is corrupted. The errors are defined in UTidyHeap.h. Verify checks the integrity of the user data by cross referencing the pointers (the user data points at the TTidyHeapBlock and the TTidyHeapBlock points at the user data), and by checking that the markers are all valid. If the verification fails, it will log the failure to Logf and return an error.

You can exempt a block from getting tracked at any time by calling 'Except' with a Boolean parameter. An exempted block will not be reported as leaking at termination time.

Examples:

Here's how to get the TTidyHeapBlock from a user pointer, and then an example of verifying the block.

CUserClass *myClass = new CUserClass;
 
TTidyHeapBlock* myBlockPtr = TTidyHeap::Instance()->GetBlock(myClass);
 
TTidyHeapError err = myBlockPtr->Verify();


TTidyHeapAction(s):

TTidyHeap has moments when optional actions may be performed. These actions are lightweight classes that are designed to modify the behavior of TidyHeap or modify the block of data that is being currently operated on. This class has a virtual Perform method whose parameter is a pointer to the block's TTidyHeapBlock class. This action class architecture is designed to allow easy customizing and addition of features.

Two types of actions:

  1. Per block: The client may optionally assign one action to each block. These actions are performed immediately before deletion (after the blocks destructor is called if it is a class).
  2. Global: Global actions are performed on all blocks at specific times. You can have as many global actions as you would like.

Available global actions:

  1. new : TTidyHeapAction::Perform is called after allocation and before the block is returned from new. Add a global new action to TidyHeap with AquireNewAction.
  2. delete: TTidyHeapAction::Perform is called immediately before deletion (inside of delete). Add a global new action to TidyHeap with AquireDeleteAction.
  3. termination: This is a special action that is called immediately before TidyHeap is deleted. This provides for post static destruct time cleanup: for example, the logfile class is deleted by one of these special actions.


Provided TTidyHeapAction classes:

These are actions you can use right out of the box.

  1. TActionBreakOnDelete:
      This action is meant to be a TTidyHeapBlock's delete action. This basically calls DebugStr and logs the break to Logf. You can see when a specific block is being deleted this way.
  2. TActionMakeNewFail:
      This action will cause new to fail when new is called x times. Send the count you want to fail into the constructor. This needs to be a global new action.
  3. TActionIncrementalNewFailer:
      #include "UActionIncrementalNewFailer.h"

      This action causes new to fail incrementally.

      Each time a TActionIncrementalNewFailer is instantiated, it saves an incremented by one count out to a MPW text file. TidyHeap saves this number as text so that you can edit it with MPW and when TidyHeap loads it in the next time it converts the text to a number and then will new will fail at that number. This text file is called 'YourApplication.cnt' and is saved in the default location. This should be a global new action.

  4. TActionDoubleDeleteWatcher
      This action causes all affected blocks to never be deleted so that we can safely look for doubly deleted blocks (Obviously, this is a debugging tool that shouldn't be run with all the time ;-). When the user calls delete TWhateverClass the block is checked for its delete flag, if it has already been deleted, and error will be logged. The block will still not be deleted.

      This can either be a global new or delete action, or it can be attached to a specific block as a delete action to test for double deletes on a specific block.

  5. TActionVerifyAll
      This action causes *ALL* blocks to be verified when the action is told to perform. This can be any type of action.
  6. TActionVerifyBlock
      This action verifies the current block being operated on. This can be any type of action.
  7. TActionBreakOnNew
      This action will break into MacsBugs when the block with the specified ID is created. The ID is specified by a parameter in it's constructor.
  8. TActionExemptBlocks
      This action exempts blocks from being reported as leaking. You can send in a ID value when creating this action which will specify the last block ID to ignore. The follow example will exempt blocks from the current ID, which is assumed to be less than 50, through block ID 50. Sending in zero (the default) exempts all blocks. Note that you can create the action and keep a pointer to the action around and when done exempting blocks, you can call "RemoveNewAction". This is an alternate method of exempting a 'range' of blocks.

      For Example:

      TTidyHeap::Instance()->AquireNewAction( new TActionExemptBlocks ( 50 ));

     


TTidyHeapState:

The TTidyHeapState class is a lightweight stack based class for doing memory leak tracking. When the class is created, it puts a marker class into the TTidyHeapBlockList. When TTidyHeapState::Verify method is called, either by client or by its destructor, the block list is checked. If there are any blocks in the block list after the state's marker, these rogue blocks are logged, and a count is returned.

Example:
...
TTidyHeapState mySnapShot;
DoSomethingThatMayLeak();
long myNumLeakedBlocks = mySnapShot.Verify();

...


Header and trailer format:

------------------
| 0xF1F1F1F1     | <-- Marker1
| 0xXXXXXXXX     | --> Pointer to block's TTidyHeapBlock
| 0xF2F2F2F2     | <-- Marker2
|----------------|
| Client Data    |
|                |
|----------------|
| 0xF3F3F3F3     | <-- Marker3
------------------

Markers:

Both the header and the trailer have special data members called "markers". These are 32 bit numbers that TidyHeap sets to values which are unlikely to occur in nature. Currently, these values are 0xF1F1F1F1, 0xF2F2F2F2, 0xF3F3F3F3. This markers are labeled in the code Marker1, Marker2, and Marker3.

Header format:

The header has two markers (Marker1 and Marker2) that delimit a pointer to the block's TTidyHeapBlock class (see below).

typedef struct BlockHeader
{
    TTidyHeapMarker fMarker1;
    TTidyHeapBlock* fBlockPtr;
    TTidyHeapMarker fMarker2;
};

Trailer format:

The trailer's only data member is one marker (Marker3). It has no other data. Its sole purpose is to provide a way to verify the integrity of the block, i.e. if the client data is an array and if someone writes past the end of the array the verification of the block will fail because Marker3 will be corrupted.

typedef struct BlockTrailer
{
    TTidyHeapMarker fMarker3;
};
 

Compilation options:

There are currently two compile time flags available.

  1. qTidyHeap

    if qTidyHeap is defined to 1 the static method TTidyHeap::Instance() is available, and all the source code is compiled (if it's included directly in your application), and all the headers are availiable.

    If qTidyHeap is not defined or is defined to 0, you can still have your all of your "new"s defined as TH_new, which is #defined to 'new' for non-debug builds. None of the headers will be available, everything is compiled out.

    Example:
     
    #if qTidyHeap
        TTidyHeap::Instance()->VerifyAllBlocks();
    #endif
  2. qDebugTidyHeap

    This turns on some internal debugging code for TidyHeap.


Creation/Deletion of TTidyHeap and its Shared Library:

TTidyHeap is created the first time the new is called. This includes new calls at static construction time. TTidyHeap is guaranteed to be created at all times, including during static create and delete times.

TTidyHeap has an accompanying shared library, whose sole reason for existence (at this writing) is to destroy TTidyHeap after static destruction time. If qTidyHeap is defined to 1, then this library is required.


TH_new:

TidyHeap provides an optional macro replacement for "new", TH_new. This macro, when used, saves the file name and line number of the call to new into the new block's TTidyHeapBlock. When a block is logged, its file name and line number will be logged as well, allowing for easy tracking of memory leaks. If the macro is not used, when logging the block, it will log "unknown" for both file name and line number.

Note: You can also call TH_SaveBlockInfo() just before new to also save the info into your block.

Example:

CUserClass *myClassPtr = TH_new CUserClass;

or

TH_SaveBlockInfo();

CUserClass *myClassPtr = new CUserClass;

Hot Tip: Use NewDropper to change all your news to either "TH_new" or "new".


Files:

Files are all saved in the same location which is specified by the TTidyHeap::Instance()->GetSessionFileLoc. The current location is in the same directory as the hosting applicatons. Files default to the name of the current TidyHeap session, which in the application world is the name of the hosting application. Files are differentiated by their extention as follows.

Currently there are three files created by TidyHeap:

  1. SessionName.prf

    This holds all the current preferences for the session of TidyHeap. This file is editable by "TidyHeap Director".

  2. SessionName.log

    This is the output from the TTidyHeapLogFileLogger. This is a text file that can be read by any Text Editor. The output is formatted in MPW scripting language. If you use TH_new you can select the file in MPW and go right to the file and line number. Note that TidyHeap never has the full path name to the file so you'll have to set the current directory in MPW to the correct directory first.

  3. SessionName.cnt

    This is the incremental failer count file. You can change the value with any text editor. For example, if you change the value to '50' your application will fail on new number 50 the next time you run it.


Tools:

Two tools ship with TidyHeap to make your life easier. Here's a quick description of both:

1) NewDropper

The tool 'NewDropper' is an application that replaces new with TH_new, or TH_new with new, within the source files dropped on it (note: it only changes text in files that are source files). It recursively searches all directories dropped on it, so you can drop your entire project directory tree on it and it will do the right thing. Of course, be sure and make a backup of all your source before using NewDropper!

2) TidyHeap Director

When TidyHeap is initialized it reads a settings file that holds different settings for TidyHeap. These settings determine how TidyHeap behaves. These settings can be edited by TidyHeap director so that you can change the behavior of TidyHeap in your application without recompiling. For example, you can have TidyHeap install your favorite TidyHeapActions at startup, or you can even turn tidyheap off. Just simply open the the settings file, titled "your application name.prf", with TidyHeapDirector. Closing the settings window saves the new settings. The settings are only loaded in when tidyheap is created in your application, i.e. changing the settings while you application is running won't have any affect until your application quits and restarts.

The following is a list of feature flags and installable actions controllable by TidyHeap Director.

Editable features:

  1. Enabled: TidyHeap on/off
  2. Warnings: Turn warnings on/off
  3. Undeleted blocks: Turn undeleted blocks warning on/off (this is the warning that comes up when you app quits)
  4. Log undeleted blocks: if this is turned on it will log a description of each undeleted block to the logger when your application quits. Note that this can be slow if you have a lot of blocks.

Global New Actions controllable by TidyHeapDirector:

  1. IncrementalNewFailer
  2. VerifyBlock
  3. VerfiyAllBlocks
  4. MakeNewFail
  5. Global Delete Actions:
  6. DoubleDeleteWatcher
  7. Verify Block
  8. Verify All Blocks


How to Hook TidyHeap into your application or framework.

There are three steps to link your application or framework into TidyHeap.

1. Build the 'TidyHeapLib' static libary with the Metrowerks project "TidyHeapLibPPC_d.¼" and add the built "TidyHeapLib" to your application or framework project.

or

1. Include the TidyHeap source files directly into your project.

2. Include your application or framework specific configuration files in your application or framework project (see below).

3. Include the "TidyHeapSharedLib" in your application's Metrowerks project.

Note: Make sure that TidyHeap's new and delete are the new and delete that are linked to. If your project has two or more libaries that define new and delete, the first one in the project will be linked to. You can insure this by moving the TidyHeap lib or source above the other implemenation in the project list.

Configuring TidyHeap for your application or framework is fairly simple and involves 3 steps:

1. Decide if you need to subclass TTidyHeap, and define it if you do.

    You can make this decision by evaluating if you need to override any of TTidyHeap's methods. Some key methods are as follows:
  • HandleNilAllocation(): This method gets called when 'new' is forced to fail by a new action. You may want to throw an exception or just do nothing, i.e. return NULL from new. In MacApp r11 we call FailNULL so that a Failure is signaled, which emulates what MacApp does when it really runs out of memory.
  • CreateBlockObject(): This is a factory method for creating each memory blocks TTidyHeapBlock. If you want to subclass TTidyHeap for your own purposes, this is how to do it.

    Example: Here is the MacApp specific subclass of TTidyHeap which ships with TidyHeap Release 2:

    class TMacAppTidyHeap : public TTidyHeap
    {
    private:
    friend TTidyHeap* CreateTidyHeap();
    	TMacAppTidyHeap();
    	virtual ~TMacAppTidyHeap();
    	
    public:
    virtual void HandleNilAllocation();
     
    };
    TMacAppTidyHeap::TMacAppTidyHeap()
    {
    }
    TMacAppTidyHeap::~TMacAppTidyHeap()
    {
    }
    void TMacAppTidyHeap::HandleNilAllocation()
    {
    	::FailNIL(NULL);
    }

2. Define the 'CreateTidyHeap' factory procedure. Note that this procedure is prototyped in UTidyHeap.h and is expected to be defined by the client. Two things need to happen in this procedure:

  • Set TidyHeap's Global New and Delete procedure pointers. Note that these default to NewPtr and DisposePtr.
  • Allocate the instance of TTidyHeap or it's subclass.
    Example: (MacApp's implementation)
    TTidyHeap* CreateTidyHeap()
    {
    	TTidyHeap::GlobalNew = MAOperatorNew;
    	TTidyHeap::GlobalDelete = MAOperatorDelete;
    	return TH_new TMacAppTidyHeap();
    }
 

3. Make sure your CreateTidyHeap and optional subclass of TTidyHeap is included in your project or TidyHeap library.