Handling Large BMP Files

When Microsoft created the BMP format more than 20 years ago, I am sure the developers never thought people would use it for applications that required several Gigapixels. So they made some implicit assumptions about the file size that prevent it from holding more than 4 GBytes of pixels in a single file.

Header Limitation

That's because the BMP header uses only 4 bytes (32 bits) to define the file size: 232 = 4,294,967,296 . So you can't enter a file size larger than 4GB using 4 bytes.

Still a Windows program should, in principle, be able to handle BMP files up to 4GB when running on a PC with enough RAM. Yet many programs fail for much smaller files -- say 120MB. Why?

Inefficient Storage of Monochrome Files

Possibly more limiting is the way many bitmap libraries handle monochrome BMP files. Even though a monochrome bmp file requires only a single bit per pixel, we've noticed that many libraries allocate a full byte per pixel in memory when loading monochrome BMPs. This greatly blows up the bitmap's footprint in memory. The limitation seems to exist in both 32 bit and 64 bit versions of Windows.

VLBV Uses Memory Compression

Artwork wrote its own underlying libraries for handling bitmaps and our library does just the opposite. We take an uncompressed monochrome bitmap and compress it while loading it into memory.

Since most of the bitmaps we use compress very highly (they are not random gradients but PCB or IC circuits) we often end up with compression ratios of between 10 and 20.

So a 2 GByte monochrome BMP file on disk produces a 200 MB footprint in memory. And that is why VLBV can handle large monochrome files that other tools choke on.

Specific Example - Adobe Photoshop

Adobe Photoshop is pretty much everyone's go-to tool for examining and manipulating bitmaps. Photoshop can handle extremely large TIFF files but when it comes to monochrome BMP it is not so capable.

In order to test Photoshop we created a series of larger and larger BMP files until we got Photoshop to fail.

The Test File

We started with a simple Gerber file that is typical of those that are printed using laser based imaging equipment. You can see it below. The panel covers slighly more than 17 x 17 inches.

snapshot of the Gerber file, contour.gbr

We then used Artwork's Gerber rasterizer to generate BMP output at different DPI's in order to get different output file sizes. The command line is shown below:

gbr2tiff.exe  contour.gbr -274x -ram:512 -aw -inch -dpi:1000 -bmp:1000dpi.bmp
gbr2tiff.exe  contour.gbr -274x -ram:512 -aw -inch -dpi:1200 -bmp:1200dpi.bmp
gbr2tiff.exe  contour.gbr -274x -ram:512 -aw -inch -dpi:1400 -bmp:1400dpi.bmp
gbr2tiff.exe  contour.gbr -274x -ram:512 -aw -inch -dpi:1600 -bmp:1600dpi.bmp
gbr2tiff.exe  contour.gbr -274x -ram:512 -aw -inch -dpi:1800 -bmp:1800dpi.bmp

which produced files of size:

Filename       Size (KB)     Row Width     Number    Adobe 
                              (pixels)     of Rows   Open?
1000dpi.bmp    37,549 KB       17,550      17,509     YES
1200dpi.bmp    54,085 KB       21,060      21,010     YES
1400dpi.bmp    73,540 KB       24,570      24,513     YES
1600dpi.bmp    96,083 KB       28,080      28,015     YES 
1800dpi.bmp   121,636 KB       31,591      31,517     NO

For the files ripped between 1000 and 1600 DPI Adobe opens the BMP file quickly:

snapshot of Adobe CS4 with dpi1600.bmp open

At 1800 DPI we produce a monochrome BMP file that is 121 MB but which Adobe Photoshop CS4 (64 bit running with 8 GB RAM) cannot open. We don't know why but once you reach a certain size you can no longer open or otherwise work with the BMP file. Here is the message that Adobe presents:

Photoshop error message when opening large BMP file

Don't believe it? Try it yourself. You can download the 1800dpi.bmp.zip file (2.1 MB zipped) and see if you can open it in Adobe.

The Very Large Bitmap Viewer has no problem loading the 1800dpi.bmp file. In fact, we ran the same file at 5080 DPI (0.5 um pixel size) which produced a file of 968,352 KB and VLBV opened this file in a few seconds:

VLBV can open a 970 GB BMP file with no problem

Of course, the upper limit, even for the VLBV, is a 4 GByte BMP file. This is because fields in the header are limited to this value.