Importpix is a free, cross-platform, open source command-line utility providing similar functionality to how Lightroom copies and organizes photos as part of its import process. It targets enthusiast and professional photographers who regularly import large volumes of photos into multiple libraries, and who use multiple image processing applications such as Lightroom, Capture One, DXO, Iridient, etc.


For Importpix documentation and installation instructions press the Learn More button.

Background and Motivation

Imagine needing to recover lost photos from 54 old hard drives before having them destroyed. This is the situation I recently found myself. How I ended up here is irrelevant to the topic at hand, but I do have a server that I backup regularly and the photos went missing due to human error and circumstances beyond my control. However, there was a strong possibility that lost photos might be stored somewhere in my collection of old drives.

Simple as it may sound, finding all the lost photos was to be a tedious task if to be done correctly. Let’s take a moment to explore the landscape. It’s not as simple as searching for photos by filename because humans don’t associate memories with filenames. I only knew memories were missing from my collection, and I couldn’t even tell you which cameras captured these memories much less the filenames. I’m also an enthusiast photographer and have been for decades, so I have a lot of photos. Over 1TB ignoring duplicates, and several terabytes in all. My old drives consisted of backups, backups of backups, system drives, and data drives — most with deeply nested directory structures. I didn’t want to make any assumptions about where the photos were stored as I often make cached copies when working with them. In a nutshell, finding the lost photos would be like looking for a needle in a haystack.

Searching for all photos and copying them into a folder was not an option because one must account for different photos having the same filename. This is inevitable if the photos were captured by two or more cameras from the same manufacturer unless steps were previously taken to ensure that each photo has a unique filename. Preserving filenames would address the problem of photos being overwritten, but this would explode the number of duplicates in my collection and generate a lot of ugly filenames. And after all that, I’d still be looking for a needle in a haystack.

This led me to develop Importpix to not only address my lost photos problem but to also improve the day-to-day workflow around how my photos are imported and organized. Organized means having a consistent directory structure for all photos with applied filename formatting rules. For example, using Importpix, I maintain a directory structure where photos are grouped according to capture date by month and year. The filenames include the capture time and an index value generated by the camera. Using this approach I know exactly where to find a photo based on its filename. Also, there is very little chance a photo getting overwritten if I ever need to copy some or all of my photos into a flat folder. I could have opted to include camera serial number information in the filename I to eliminate any chance of collisions but opted for shorter filenames instead.

Why not just use Lightroom?

Importpix is not a replacement for Lightroom. I don’t want there to be any confusion about this so, again, Importpix is not a replacement for Lightroom. In fact, Lightroom is the primary application I use to process photos and I have no intention of abandoning Lightroom anytime soon. With that out of the way, you might be thinking why not just use Lightroom? I thought about this too. After all, again, I’m already using it to process photos and organizing photos is a big part of what Lightroom does so it seems like a natural choice. But I opted for a different solution for several reasons.

The primary reason is I didn’t want to take a dependency on heavyweight image processing application like Lightroom for the basic structure of how my photos are organized. This structure should never have to change and I’d give serious consideration to using Capture One instead should Phase One ever add support for third-party medium format cameras. I decided it’s best to factor out the basic structure of how my photos are organized from photo processing applications like Lightroom and Capture One. This gives me the flexibility to change applications without affecting this structure.

Secondly, Lightroom is not optimized for the lost photos recovery scenario described above. It performs reasonably well when importing hundreds or even thousands of photos from an SD card. But if the number of files becomes very large then import performance can degrade to the point of rendering Lightroom unusable, especially if high-resolution RAWs are involved. If this resonates with you, then imagine pointing Lightroom at your entire backup collection and hitting the import button. For the rest of you, it’s not pretty and I needed a solution that could accommodate this task.

Finally, wanting all my photos organized doesn’t mean wanting every photo ever taken imported into Lightroom. I could import the photos and then remove from without deleting from storage, but that creates more work after what would surely be an excruciatingly long import process. This is a lot of effort when needing to unwind something I didn’t want in the first place, just to leverage a subfeature of Lightroom’s import process.

Introducing Importpix

Importpix is a lightweight command-line utility that uses EXIF metadata to organize photos as part of a file copy operation. The source is typically removable storage in the form of an SD, XQD, CFast, or CompactFlash card, but it can be any path accessible by the host, including a network share. The target folder or destination is called a library, and the only thing that separates a library from an ordinary folder is a hidden file at the root defining the rules for how photos should be organized within the folder. Just like the source, a library can be any folder path accessible to the host. The rules are expressed as templates which get expanded using the EXIF metadata for each photo processed. The expanded forms determine the location and filename of the photo within the library. The following snippet shows how templates are defined:

   "templates" : {
      "filename" : "P{[DATE1]}-{[HHMMSS]}-{[INDEX]}",
      "layout" : "{[YYYY]}#{[MM]}"
   }

The filename template defines the filename rules for library photos, and layout defines how they are organized. The template values shown are largely self-explanatory and can be customized for user preference at the time the library is created. In this example, the library is configured to organize photos by year and month. The ‘P’ at the head of the filename template starts each filename with the letter ‘P’ (for photo), which is my preference, but not required. Refer to the documentation for full details regarding these and other token values.

Don’t let the naming of Importpix fool you into thinking its something more complicated than it actually is. Unlike Lightroom, there is no importing of information into databases or catalogs. Its stateless operation really is just about copying files with applied logic for organizing photos, and with the following features and benefits:

  • Free and open-source software
  • Organizes photos and images using Exif metadata
  • Handles duplicate detection
  • Preserves file groupings (see explanation below)
  • Handles XMP sidecar file remapping
  • Supports unlimited, user-defined libraries
  • Stateless design with no database or service daemon required
  • Photos and images are not modified
  • Works on Linux, MacOS, and Windows
  • Leverages Exiftool for reading Exif metadata from photos

Most of the features are self-explanatory but the preserving file groupings feature requires explanation. It’s common for photographers to shoot RAW+JPEG, which produces two files having the same basename with different extensions. If sorting files by filename, they will appear together since they have the same basename. However, different file formats don’t always share the same Exif metadata properties. Hence, if a person specifies an import rule based on a property that exists in only one of the formats then they would have different basenames after being imported. Importpix addresses this problem so that the files continue to appear together after being imported. Importpix similarly addresses the problem of edited files on disk, when the edited version’s basename is derived from the original.

I’ll also touch briefly on the duplicate detection feature. It prevents duplicate photos from being imported into a library, thereby saving storage and for a clutter-free photo management experience. The motivation behind this feature was the lost photos recovery scenario described above, where I rebuilt my photo library by scanning all of my storage devices (including the old drives) for all photos. Since the devices included backups (and backups of backups), there’d be a lot of duplicates. A surgical approach would require a lot of my time and is error-prone. A blunt-force instrument was needed where I could point it at a drive (or folder) and rely on it to import everything modulo the duplicates. It might take a lot of computing time, but it’s something I could let run unattended with the comfort of knowing that no photo would be left behind. Of course, this blunt-force instrument became Importpix.

For more details regarding the other features, refer to the documentation.

Using Lightroom with Importpix

If you’re a Lightroom user like me, you can potentially take advantage of Importpix to improve your photo management workflow with only minor adjustments. The key concept is to think of Importpix as the tool that gets your photos copied to and laid out on disk according to your preferences. In doing so, you’re freeing Lightroom from needing to do this work. This has a few advantages over using just Lightroom:

Faster import times

Lightroom import times are faster because the work of copying the files has been offloaded to Importpix. Naturally, the files must still be copied into the library so the total amount of time spent is roughly the same. The difference is that by using Importpix the copying happens is as a separate process, so Lightroom is virtually unaffected while this work takes place. Once complete you still need to import the photos into Lightroom, but it now has much less work to do. This may or may not be an advantage depending on how much time you spend in Lightroom. If find yourself waiting a lot on Lightroom imports to complete, then you’ll find this approach useful.

All photos organized the same way

I don’t import all of my photos into Lightroom. For example, I have several thousands of time-lapse photos that I don’t need or want in Lightroom. Importpix makes it so you can organize all of your photos the same way, irrespective of your decision to include some of them in Lightroom or not.

Fast offloading of photos from SD cards

If your only goal is to get the files copied to your workstation for safe keeping (or to free up your SD card to take more photos), then you’ll find Importpix to be faster than using Lightroom because it just copies files. If multiple SD cards are involved and your workstation has a multislot SD card reader then you can also copy the files in parallel. This can greatly speed up the total amount of time spent. When you’re ready to import the photos into Lightroom, they’ll already be copied into place and you’ll have the added benefit of all the files being part of your previous import collection if the photos came from multiple SD cards (or multiple trips by a single SD card).

The following examples illustrate all that required to start the import process. The examples assume a default library has been configured for Importpix, else a library target option would also need to be specified as a command-line option.

MacOS and Linux:

If the source path to media containing files to be imported is /Volumes/Untitled then open a terminal window and enter the following command:

importpix /Volumes/Untitled/

Windows:

If the source path to media containing files to be imported is E:\ then open a command window and enter the following command:

importpix e:\

More Information and Installation Notes

For Importpix documentation and installation instruction visit the following link:
http://rolanday.github.io/importpix