sgbotsford

I have tried Libre Office and Open Office. In some cases I found them 'bug compatible' -- the same problems I had in Word, I had in them. I'm not impressed. When I tried their database application -- purportedly a replacement for Access, I averaged about 3 minutes between crashes. OTOH I've yet to have major issues with Google Docs opening Word doc and docx files. nor Google Sheets opening excel sheets. True: Not all functionality survived the round trip. Word does a lot of things that Docs doesn't. And Excel has functions and formatting features lacking in Docs. By comparison there are MANY apps that can both read and write PSD files. Likely also with things that don't survive the round trip.

Owning market share isn't sufficient for me to gamble decades of work. No one makes an adequate one photographer, small photography shop PDAM.

As an example of the flip side, look at Microsoft Docx format. Everyone can read it. Many apps can write some form of it. Ditto Excel. They aren't losing market share. Adobe is a crap vendor. Their user interface is inconsistent, software buggy. Photoshop mangles metadata it doesn't understand. Every mac owner knows to their pain to not install a new OS-X version for half a year after it comes out because Adobe apps will break. (We've been down that road with InDesign.) Their product support is awful. (See FrameMaker; Flash) AFAIK Illustrator uses a subset of postscript as it's internal format. And PS is mostly a write only format. There are many ways to do things. (In fact PS is an actual programming language -- it's Turing Complete. In principle you can write an operating system in postscript. )

I've said this before, but it bears repeating: Introduction Currently using Apple Aperture. Need a replacement. I've been thinking a lot about photo management. I'm starting to avoid the word 'DAM' as it increasingly refers to industrial sized software costing tens or hundreds of thousands of dollars. So let's look at what I mean by a photo manager: Browser -- look at a bunch of pictures Tagger -- add metadata either singly or in batches. Searcher -- use complex searches to narrow down what I look at. Version tracker -- ability to keep track of derived images. That's the TL;DR version. Next level of detail: Browser Pix initially come in in bunches, and as such they go somewhere in some folder structure on your computer. Many people will use some combination of Year/Month/ and string to describe event. Often remembering that the shot took place on your trip to Italy, or that it was the Smith & Brown wedding is sufficient. Tagger But what do you do when you are looking for the closeup of a butterfly. It was an incidental pic on some holiday, but which one. Now metadata comes into play. If all your holiday shots are tagged 'Holiday' and your program can search existing metadata, your problem is solved. Search for holiday and focal distance less than 5 feet. You still may have a bunch to wade through. If, in addition to some general keywords for the batch you add a few per image you have a big win. E.g. "Butterfly" Tagging is hard. You want to tag it with multiple things. E.g: Describe the scene. Identify the location. GPS is fine, but "Lock Ness, Scotland" or Kensington market, Toronto, Ontario, Canada is easier to visualize. ID the people in the scene. Classify them more generally. (Woman with child; Young boy...) Describe the technical aspects -- close up, high/low key, lighting One or more classes of description about the scene -- weather, mood. Usage: Have you sold exclusive rights for this image? Exclusive for 18 months for a calendar? These are facets. I prefer to go through a set of images several times, concentrating on one of these at a time. Sometimes a facet is irrelevant. Weather makes no sense for an interior shot. If you do facets, you need a way to search for images that don't have an entry for facet X. You also need a way to mark a facet as irrelevant. Crudely you can implement these with constructs like WEA:Cloudy but then you still have to be able to search for images that don't have WEA:* as a keyword. And you have to decide on what to do where it's not relevant. WEA:N/A Having some kind of support for actual facets would be a big win. Hierarchical keywords are important, and it's partner, controlled vocabulary. You really want to be able to avoid having entries for Smith, John and be unable to distinguish between the one from Hoboken, NY and the one from East Horsebiscuit, SD. You also want to avoid Rachmaninov, Sergie and Rachmaninoff, Sergie. So controlled vocabulary is your friend. At the very least it should require you to take an extra step to add a new word. The database needs to be bulk editable. E.g. When you started out you had a category, "People" and everyone was under people. Well, after a while that was getting cumbersome. So many friends. So you want to introduce some subcategories People -> High School Friends; people -> industry aquaintences... You want to be able to move someone from one category to another, and have those changes propagate to the images involved. Searcher No point in tagging if you can't search the data. Two programs I trialed, Mylio and Photoshop Supreme, had no provision to search exif data -- where the stuff like time of day, and focal length, and camera model is kept. One program allows you to search for only one tag at a time. I can search for Holiday. Or I can search for butterfly. But I can't search for shots that have both "Holiday" and "butterfly" Ideally you want full boolean search support with 'and,' 'or', & 'not', parentheses for grouping, and wild cards for partial matches. Version tracker A photograph for a professional may have a long history. You often have a shot, then export it in some altered form (cropped, resized, sharpened, colour adjusted, watermarked) Nice to be able to find the original 5 years from now. One recommended practice I ran into had the following: Master image was Raw. Archive version was digital negative. Processing version was 16 bit tiff or PSD Delivered version was tiff or jpeg. This requires a minimum of 4 versions. Add to that: Watermarked versions. Reduced resolution versions for web pages. Colour matched versions for specific printing environments. Cropped versions for mobile web pages. So that's the base case. Implementations differ, and they refine this somewhat. Online resources Impulse Adventure (site: https://www.impulseadventure.com/photo/) Unfortunately out of date. But still several good articles. Catalogs and Multiple versions. https://www.impulseadventure.com/photo/flow-catalog-versions.html Important Features of Catalog Software. https://www.impulseadventure.com/photo/flow-catalog-features.html Controlled Vocabulary (site: https://www.controlledvocabulary.com/ ) Using Image Databases to Organize Image Collections http://www.controlledvocabulary.com/imagedatabases/ Also has a good forum/mailing list. Requirements: The four functions above describe what it should do. Here are some more details about how it should do it. Server requirements I can see implementing this in one of two ways: Either as a stand alone program or as a local web server. The latter has the advantage that it would scale for family or small photo business. Cloud services are slow when you are talking about 10-12 Mbyte files. My network connect takes several seconds per MByte. Cloud services for metadata have to be well optimized -- you really don't want to issue 3000 keyword change requests individually when you change the spelling of a keyword. So: Not cloud based. Runs on Mac or on local apache web server. Keyword handling Fast keywording. Aperture allows drag and drop from a list, multiple sets of hotkeys for words used frequently, copy paste of keywords from one photo to another, and keywords organized in folders. It also allows search for a keyword, and a list matching what you typed so far appears. Other programs that have good keywording include IMatch and Photomechanic. One of the key aspects of this is to have multiple ways to do things. I like aperture's multiple preset buttons -- combine with facets. A history of keywords might help: A pane with the last N keywords in it. Chances are that the next word I use will be one of the last 20 I use about 80% of the time. Full access to standard metadata: EXIF, ITPC, subject to limits of the file format. Controlled vocabulary. I want an extra step to add a new keyword to my list of keywords. This helps with the the Sommer Vacashun problem. Hierarchial vocabulary. E.g. Separate entries for Birds -> raptors -> falcon and Planes -> fighters -> falcon. Parents are stored with keywords. Moving a keyword in the master list, or changing spelling, corrects all usage in photos. This can be done as a background task. Parent items are automatically entered as keywords. (With the correct database linkage, this comes free as a side effect of the point above. Synonyms -- I can define "Picea glauca" as a synonmym for "White Spruce" entering one, enters the other. Facets: For a set of pictures I want to be able to define a set of facets or categories for collections or folders. Facets would be things like: Weather; Who; Where; Ecosystem; Season; Lighting Not all collections would have all facets, but a collection having a facet would nag me to put it in. A facet would have a negation for not applicable (Weather isn't applicable inside a house; Who isn't applicable in a landscape shot.) Facets allow me to go through a collection in multiple passes and get the missing keywords. Searching Complex searches: Find all shots between 2012 and 2015 shot in December or January, shot with my Nikon D70, with keyword "snow" rating of 3 or better shot after 3pm in the day. (Yes, I do use searches like that) Saved Searches. These are the equivalent of smart albums in Aperture. As new pix meet the standards they would be shown. Version Tracking Version tracking If a lower resolution, cropped, photoshopped, composited or a black and white image is produced from a master, the system should show that it's a derived image, and allow access to the master. A master should be able to list derived images. Derived images are not linear but form a multi-branched tree. If my camera produces JPEG and Raw versions, I want the JPEG to be shown as being derived from the Raw version. Metadata applied to a master should propagate down to derived images. Some form of exception handling for this: e.g. -keyword to prevent a people identifier being applied to an image where that person was cropped out. Ability to track through external editing programs. E.g. If I edit a program in photoshop, it will mark the PSD file as being derived, restore as much of the metadata as the PSD format allows. If Photoshop is used to create a jpeg image, that too is tracked. Data robustness All metadata is indexed. Metadata is also written to sidecar files. Where possible metadata is written to the image file itself. (optional -- can stress automated backup systems) Through file system watching, name changes and directory reorganization are caught. Relevant sidecars are also renamed, and the database updated with new file location/name. Sidecar contents include the name of their master file. Should be possible to rebuild entire database from images + sidecars. Should be able to restore all file metadata from database. This requires a lot of under-the-hood time stamps to determine which has priority. All database actions should be logged and journaled, so they are reversible. Reasonable speed with catalogs of more than 100,000 images. Support for previews of all common image formats and most raw formats. Previews and thumbnails are treated as versions of the master. They inherit metadata. Nice to have: Simple non-destructive editing -- crop, brightness, contrast. Rating system Smart albums Drag and drop functionality with other mac apps. Metadata Storage There are three places metadata can be stored: In the image. In a database. In a separate file for each image (sidecar file) Typically these files have the same name as the primary file, but a different suffix. If at least some cataloging information is written to the image, then you can reconnect a file to your database. In principle this can be a single unique ID. This saves you from: You moved or renamed an image file. If you can write more info into the file -- keywords, captions -- then you are saved from: Your database is corrupted. You upgraded your computer and your database program doesn't work there. Sidecar files allow you to recover all your metadata if your database crashes. Downsides of storing data in the image Writing to the original files can corrupt the file. Most RAW formats are well understood enough now to at least identify and replace strings of metadata with the same length string. If you tell your camera to put the copyright string Copyright 2018 J. Random Shutterbug Image XXXX-XXXX-XXXX-XXXX-XXXX-XXXX Then as long as the DAM keeps that string the same length you are golden. Keeping all metadata (or as much as you can) in the original images makes for very slow access. Your program has to read at least the first few blocks of every image. Depending on the file structure, adding too much data may require rewriting the entire file. Any program that moves the boundaries of data sub blocks better be well tested. Writing data back is time consuming. Some file formats don't have any metadata capability. Some file formats (Photoshop PSD) are noted for mangling metadata. A glitch during the write process can corrupt the image file. The alternative, writing a new file, then replacing the old file requires that the entire file be both read and written, rather than just a chunk of it. This has serious performance issues. Downsides of Databases Databases are fast, but they are blobby, and you are writing into the middle of blobs of data. If the implementation of the database is solid, there isn't much to worry about. But hard disks have errors, and a single error can make a database partially or fully unusable. Good database design has redundancy built in so that you can repair/rebuild. Databases are frequently proprietary. Data may be compressed for speed. Getting your data out may be tricky. (Problem for people using Apple Aperture) Databases frequently are optimized in different ways. In general robustness is gained at the cost of performance and complexity. One compromise is to write all changes first to a transaction file (fast...) and then a background process does the database update in the background. This slows down access some: Have to check both the main database and the transaction file, but unless the transaction file gets to be bigger than memory, this shouldn't be noticeable. Downsides of Sidecars You have to read a zillion files at startup. If you do a batch change (Add the keyword "Italy" to all 3000 of your summer holiday trip shots) the catalog program has open, modify and write back 3000 files. If you rename a file, and don't rename the sidecar file too, your meta data is no longer connected to your image. Best practice Opinion only here: Sorry. You want a unique asset tag that resides in the image. This can be an actual tag like the copyright one mentioned above, or it can be a derived tag from information in the image. This could be the EXIF time stamp (Not unique -- multiple shots per second, multiple cameras.) If your program reads makernotes, the best one is Camera model + Camera serial number + timestamp + hundredths of a second. You want a database for speed. It, of course has the unique ID You want sidecars for rebuilding your database, and for data portability. They have the unique ID. If the database crashes, it can be rebuild from the sidecars. If a sidecar is corrupted, it can be rebuilt from the database. If an image is renamed the ID can be used to reconnect it to the sidecar, and to fix the database. To make this work, you have to use a lot of timestamps. If the sidecar is more recent than the latest time stamp in the database record, then the sidecar is the authoritative record. You also have to have internal checks on data integrity. The record for an image (sidecar or database) needs a checksum to verify that that data isn't corrupt. Given the relatively fragile nature of raw files, best practice is a system that only writes zero or once to the Raw file. This is why the exif time stamp + hundredths, copyright work well. You can include the camera model and serial number in the copyright so that now the copyright message is unique to the camera. At this point you have the ability to create, and recreate a unique ID for each image. If the DAM has the ability to modify the file, you can create this ID once. This saves some time if you ever have to rebuild the database. Having as much of the metadata in the file as possible means taht it travels with the file. This is a win, but comes with the risk of potential corruption. Possibly the best strategy is to leave the original intact, and for clients who need raw data, either add metadata to a copy, or to a derived full data equivalent (e.g. DNG) Sidecars don't need to be updated in real time. The slick way to do this would be that whenever the database makes a change to a record: Make a new record that duplicates the old record in the database. Make the change in the new record. New record is flagged, "not written to sidecar" Old record is marked "obsolete" Another thread writes the sidecar files out, writing out the new one, then deleting the old one (or renaming the new one to the old one's name). Periodically you run a cleanup on the database removing obsolete records older than X days. This gives you the ability to rollback changes. This is not complete: It doesn't address the issue of non-destructive edits. Many programs now allow the creation of multiple images from the same master file, and do not create a new bitmap, but rather a file with a series of instructions for how to make the image from the master. AFAIK all such methods are proprietary. This results in a quandary as the apps that do a good job of tracking metadata may not be able to deal with the non-destructive edits. This can be critical if you crop a person out of an image, crop to emphasis a different aspect, and receive a different caption, etc. The workaround is that you always write out a new bitmap image from a serious edit. Ideally you have a script that looks for new NDEs and writes out an image based on this, copying the metadata from the master and at some point bringing it up for review for mods to the metadata. Robustness against external programs. I like having an underlying file structure organization. I like the idea that if I produce a bunch of cropped, watermarked, lower resolution, etcetera versions of an image that my catalog will track that too. But if the underlying file structure is exposed to Explorer or Finder, then you have the risk of a file being renamed or moved, and the database is no longer in sync with your file system. To budnip answers of the form "This is impossible" here's how to "Finder-proof" your image database. When an image is edited, a file system watcher notes that the file was opened. The file goes onto the 'watch' list. (the program fswatcher does this on mac. I use it to update my web page when my local copy has been edited.) When a new file appears in a monitored directory tree, it's noted. When a file is closed, this is also noted. If there has been a new file created it is checked for metadata. If the new file's metadata has a match for an existing file, then existing file metadata is used to repopulate missing data in the file. (Photoshop is notorious for not respecting all metadata.) Database is updated with the new file being marked as derivative of the original file. optionally a suffix may be added to the new file's image number, showing whether it derives directly from the original or from another derivative. To make this work, the two components are a unique id that can be calcuated from the master, and a file system monitor program that catches create, move, change, and rename events. Notes on current state of the art: Nothing I've found supports version tracking, especially through an external program. Lightroom and Aperture both support a type of versions -- different edits on same master, but at least Aperture doesn't copy metadata to a new version. Aperture supports Stacks -- a group of related pictures. Lightroom: Doesn't support PNG, very clunky interface, slow on large catalogs; Mylio home version doesn't support hierarchical keywords; doesn't index exif information, does not allow or syntax for searches, Photomechanic is fast for keywording and culling, but has very limited search capability. IMatch. Possible contender, Requires MS windows box. Photo Supreme: Erratic quirks. Crashes. One man shop. Can't search Exif in useful way. Fotostation: AFAIK no underlying database. Has to read metadata from images/sidecar files on startup. Slow after 10K images. (They have server based software too that is big bucks.) Luminar: A DAM has been promised Real Soon Now, but no demos, storyboards or feature lists have been published. There is a claim that it is in beta, but no one on their fairly active forum will admit to being part of the beta group. Affinity: Similar to Luminar. Commandline tools Much of the special features for version tracking could be implemented with scripts using calls to these programs. ImageMagick -- good for whole-image conversions, also can read/write internal metadata and sidecars. Exiftool -- read/write exif data reads most makernotes. fswatch -- not really an image processor, but hooks into the operating system and can alert when files have changed -- modified, renamed, moved. Enterprise level There are a raft of these with vaguely defined abilities and very high price tags. Most are SaaS and cost hundreds to thousands of dollars per month. Once I saw a large price tag, I stopped considering it. WebDAM No real information about capabilities on web site. Extensis. Expensive. Bynder. Joke program. Cloud based set of shoeboxes. WIDEN. Cloud only. Asset Bank. Starts at $500/month for up to 50 users.

Proprietary formats is a good way to fail as a business. There are a ton of programs that can read and write Photoshop (.PSD) files. If Affinity is serious about being the new player on the block, it MUST publish an API that at least allows software developers to read the format: Level 1. Read the file's meta-data, and the preview image. Level 2. Write new metadata. Level 3. Extract a version of the edited file. *** I bought affinity on a sale with the promise of a DAM. I haven't used it, because the DAM isn't there yet. Level's 1 and at least partial support for level 2 (Write *some* metadata info -- modify existing metadata) I participated heavily with Luminar's forum too. I'd learned from my lessons, however and didn't buy the software. They too are going down the "Next year..." path.

Not so. A folder structure only gives you one way to select. "I remember a festive dinner with Aunt Edna. I think it was Christmas, no Thanksgiving" "No John it was easter" "What year? "I think it was 4 years ago. Might have been 5" "Oh, at least 5, Uncle Bill was still with us." So if you have date organized folders you now have to check 3 different months in 3 different years. Yuck. In a decent DAM you can search for People:Aunt Edna & Event:Holiday Dinner

The big reason for a DAM is re-use. It's nice to take pictures, and turn some of them into stellar pictures, but once you have over a few tens of thousands it gets hard to find them. I probably spend 3-5 times as much time assigning keywords to images as I do editing them. Mind you, my use is mostly web pages, so my editing sequence is 'crop, adjust exposure, resize,' Even so, with all my keywording, I often still have trouble finding things -- due to insufficient keywording. The two really big features for me are mechanisms for fast keywording, and mechanisms for doing complex searches. So far Aperture is the least bad of all the ones I've used. I'm not a pro. A pro has an extra can of worms. They have the whole 'version' thing. It's not unreasonable for a stock photo to have several resolutions, black and white, images tuned for a particular printer, plus the whole what rights have I sold to who on a particular image. However even an amateur will often have Raw file Jpeg file (if shooting both) Photoshop or affinity file Tiff or jpeg output file. It's nice if you can easily find the original to the one you used on insta-gram. *** What are the components of a good DAM? Database for storing metadata -- This should be straight forward. There are several good ones out there that are opensource. MySQL, PostGres, come to mind. Browser -- this must be fairly easy as there are metric tons of them out there. UI -- This, I think, is the tough part. It needs to be able to do a bunch of tasks -- edit metadata singly and in batch, search, store search results. *** I bought Affinity because I thought it was going to have a DAM. I don't use it. I can't remember the last time I opened it. I've gone back to Aperture. I've tried PhotoSupreme, Media Pro, ACDSee, Bynder (what a joke). I've come to the conclusion that the DAM is more critical to me than the ability to edit.

Skylum is in trouble over it's vapourware DAM. Supposedly in beta, but although I was 'on the list' to beta test it, I was never contacted, and I can't find anyone on their forums who will admit to beta testing. Further they won't release screen shots, feature lists etc. This is what I want a DAM to do: Introduction Currently using Apple Aperture. Need a replacement. I've been thinking a lot about photo management. I'm starting to avoid the word 'DAM' as it increasingly refers to industrial sized software costing tens or hundreds of thousands of dollars. So let's look at what I mean by a photo manager: Browser Pix initially come in in bunches, and as such they go somewhere in some folder structure on your computer. Many people will use some combination of Year/Month/ and string to describe event. Often remember that the shot took place on your trip to Italy, or that it was the Smith & Brown wedding is sufficient. Tagger But what do you do when you are looking for the closeup of a butterfly. It was an incidental pic on some holiday, but which one. Now metadata comes into play. If all your holiday shots are tagged 'Holiday' and your program can search existing metadata, your problem is solved. Search for holiday and focal distance less than 5 feet. You still may have a bunch to wade through. If, in addition to some general keywords for the batch you add a few per image you have a big win. E.g. "Butterfly" Tagging is hard. You want to tag it with multiple things. E.g: Describe the scene. Identify the location. GPS is fine, but "Lock Ness, Scotland" or Kensington market, Toronto, Ontario, Canada is easier to visualize. ID the people in the scene. Classify them more generally. (Woman with child; Young boy...) Describe the technical aspects -- close up, high/low key, lighting One or more classes of description about the scene -- weather, mood. Usage: Have you sold exclusive rights for this image? Exclusive for 18 months for a calendar? These are facets. I prefer to go through a set of images several times, concentrating on one of these at a time. Sometimes a facet is irrelevant. Weather makes no sense for an interior shot. If you do facets, you need a way to search for images that don't have an entry for facet X. You also need a way to mark a facet as irrelevant. Crudely you can implement these with constructs like WEA:Cloudy but then you still have to be able to search for images that don't have WEA:* as a keyword. And you have to decide on what to do where it's not relevant. WEA:N/A Having some kind of support for actual facets would be a big win. Searcher No point in tagging if you can't search the data. Two programs I trialed, Mylio and Photoshop Supreme, had no provision to search exif data -- where the stuff like time of day, and focal length, and camera model is kept. One program allows you to search for only one tag at a time. I can search for Holiday. Or I can search for butterfly. But I can't search for shots that have both "Holiday" and "butterfly" Ideally you want full boolean search support with 'and,' 'or', & 'not', parentheses for grouping, and wild cards for partial matches. Version tracker A photograph for a professional may have a long history. You often have a shot, then export it in some altered form (cropped, resized, sharpened, colour adjusted, watermarked) Nice to be able to find the original 5 years from now. One recommended practice I ran into had the following: Master image was Raw. Archive version was digital negative. Processing version was 16 bit tiff or PSD Delivered version was tiff or jpeg. This requires a minimum of 4 versions. Add to that: Watermarked versions. Reduced resolution versions for web pages. Colour matched versions for specific printing environments. Cropped versions for mobile web pages. So that's the base case. Implementations differ, and they refine this somewhat. Online resources Impulse Adventure (site: https://www.impulseadventure.com/photo/) Unfortunately out of date. But still several good articles. Catalogs and Multiple versions. https://www.impulseadventure.com/photo/flow-catalog-versions.html Important Features of Catalog Software. https://www.impulseadventure.com/photo/flow-catalog-features.html Controlled Vocabulary (site: https://www.controlledvocabulary.com/ ) Using Image Databases to Organize Image Collections http://www.controlledvocabulary.com/imagedatabases/ Also has a good forum/mailing list. Requirements: The four functions above describe what it should do. Here are some more details about how it should do it. Server requirments I can see implementing this in one of two ways: Either as a stand alone program or as a local web server. The latter has the advantage that it would scale for family or small photo business. Cloud services are slow when you are talking about 10-12 Mbyte files. My network connect takes several seconds per MByte. Cloud services for metadata have to be well optimized -- you really don't want to issue 3000 keyword change requests individually when you change the spelling of a keyword. So: Not cloud based. Runs on Mac or on local apache web server. Keyword handling Fast keywording. Aperture allows drag and drop from a list, multiple sets of hotkeys for words used frequently, copy paste of keywords from one photo to another, and keywords organized in folders. It also allows search for a keyword, and a list matching what you typed so far appears. Other programs that have good keywording include IMatch and Photomechanic. One of the key aspects of this is to have multiple ways to do things. I like aperture's multiple preset buttons -- combine with facets. A history of keywords might help: A pane with the last N keywords in it. Chances are that the next word I use will be one of the last 20 I use about 80% of the time. Full access to standard metadata: EXIF, ITPC, subject to limits of the file format. Controlled vocabulary. I want an extra step to add a new keyword to my list of keywords. This helps with the the Sommer Vacashun problem. Hierarchial vocabulary. E.g. Separate entries for Birds -> raptors -> falcon and Planes -> fighters -> falcon. Parents are stored with keywords. Moving a keyword in the master list, or changing spelling, corrects all usage in photos. This can be done as a background task. Parent items are automatically entered as keywords. (With the correct database linkage, this comes free as a side effect of the point above. Synonyms -- I can define "Picea glauca" as a synonmym for "White Spruce" entering one, enters the other. Facets: For a set of pictures I want to be able to define a set of facets or categories for collections or folders. Facets would be things like: Weather; Who; Where; Ecosystem; Season; Lighting Not all collections would have all facets, but a collection having a facet would nag me to put it in. A facet would have a negation for not applicable (Weather isn't applicable inside a house; Who isn't applicable in a landscape shot.) Facets allow me to go through a collection in multiple passes and get the missing keywords. Searching Complex searches: Find all shots between 2012 and 2015 shot in December or January, shot with my Nikon D70, with keyword "snow" rating of 3 or better shot after 3pm in the day. (Yes, I do use searches like that) Saved Searches. These are the equivalent of smart albums in Aperture. As new pix meet the standards they would be shown. Version Tracking Version tracking If a lower resolution, cropped, photoshopped, composited or a black and white image is produced from a master, the system should show that it's a derived image, and allow access to the master. A master should be able to list derived images. Derived images are not linear but form a multi-branched tree. If my camera produces JPEG and Raw versions, I want the JPEG to be shown as being derived from the Raw version. Metadata applied to a master should propagate down to derived images. Some form of exception handling for this: e.g. -keyword to prevent a people identifier being applied to an image where that person was cropped out. Ability to track through external editing programs. E.g. If I edit a program in photoshop, it will mark the PSD file as being derived, restore as much of the metadata as the PSD format allows. If Photoshop is used to create a jpeg image, that too is tracked. Data robustness All metadata is indexed. Metadata is also written to sidecar files. Where possible metadata is written to the image file itself. (optional -- can stress automated backup systems) Through file system watching, name changes and directory reorganization are caught. Relevant sidecars are also renamed, and the database updated with new file location/name. Sidecar contents include the name of their master file. Should be possible to rebuild entire database from images + sidecars. Should be able to restore all file metadata from database. This requires a lot of under-the-hood time stamps to determine which has priority. All database actions should be logged and journaled, so they are reversible. Reasonable speed with catalogs of more than 100,000 images. Support for previews of all common image formats and most raw formats. Previews and thumbnails are treated as versions of the master. They inherit metadata. Nice to have: Simple non-destructive editing -- crop, brightness, contrast. Rating system Smart albums Drag and drop functionality with other mac apps. Suggestions? Notes on current state of the art: Nothing I've found supports version tracking, especially through an external program. Lightroom and Aperture both support a type of versions -- different edits on same master, but at least Aperture doesn't copy metadata to a new version. Aperture supports Stacks -- a group of related pictures. Lightroom: Doesn't support PNG, very clunky interface, slow on large catalogs; Mylio home version doesn't support hierarchical keywords; doesn't index exif information, does not allow or syntax for searches, Photomechanic is fast for keywording and culling, but has very limited search capability. IMatch. Possible contender, Requires MS windows box. Photo Supreme: Erratic quirks. Crashes. One man shop. Can't search Exif in useful way. Fotostation: AFAIK no underlying database. Has to read metadata from images/sidecar files on startup. Slow after 10K images. (They have server based software too that is big bucks.) Luminar: A DAM has been promised Real Soon Now, but no demos, storyboards or feature lists have been published. There is a claim that it is in beta, but no one on their fairly active forum will admit to being part of the beta group. Affinity: Similar to Luminar. Commandline tools Much of the special features for version tracking could be implemented with scripts using calls to these programs. ImageMagick -- good for whole-image conversions, also can read/write internal metadata and sidecars. Exiftool -- read/write exif data reads most makernotes. fswatch -- not really an image processor, but hooks into the operating system and can alert when files have changed -- modified, renamed, moved. Enterprise level WebDAM No real information about capabilities on web site. Extensis. Expensive. Bynder. Joke program. Cloud based set of shoeboxes. WIDEN. Cloud only. Asset Bank. Starts at $500/month for up to 50 users. Metadata Storage There are three places metadata can be stored: In the image. In a database. In a separate file for each image (sidecar file) Typically these files have the same name as the primary file, but a different suffix. If at least some cataloging information is written to the image, then you can reconnect a file to your database. In principle this can be a single unique ID. This saves you from: You moved or renamed an image file. If you can write more info into the file -- keywords, captions -- then you are saved from: Your database is corrupted. You upgraded your computer and your database program doesn't work there. Sidecar files allow you to recover all your metadata if your database crashes. Downsides of storing data in the image Writing to the original files can corrupt the file. Most RAW formats are well understood enough now to at least identify and replace strings of metadata with the same length string. If you tell your camera to put the copyright string Copyright 2018 J. Random Shutterbug Image XXXX-XXXX-XXXX-XXXX-XXXX-XXXX Then as long as the DAM keeps that string the same length you are golden. Keeping all metadata (or as much as you can) in the original images makes for very slow access. Your program has to read at least the first few blocks of every image. Depending on the file structure, adding too much data may require rewriting the entire file. Any program that moves the boundaries of data sub blocks better be well tested. Writing data back is time consuming. Some file formats don't have any metadata capability. Some file formats (Photoshop PSD) are noted for mangling metadata. A glitch during the write process can corrupt the image file. The alternative, writing a new file, then replacing the old file requires that the entire file be both read and written, rather than just a chunk of it. This has serious performance issues. Downsides of Databases Databases are fast, but they are blobby, and you are writing into the middle of blobs of data. If the implementation of the database is solid, there isn't much to worry about. But hard disks have errors, and a single error can make a database partially or fully unusable. Good database design has redundancy built in so that you can repair/rebuild. Databases are frequently proprietary. Data may be compressed for speed. Getting your data out may be tricky. (Problem for people using Apple Aperture) Databases frequently are optimized in different ways. In general robustness is gained at the cost of performance and complexity. One compromise is to write all changes first to a transaction file (fast...) and then a background process does the database update in the background. This slows down access some: Have to check both the main database and the transaction file, but unless the transaction file gets to be bigger than memory, this shouldn't be noticeable. Downsides of Sidecars You have to read a zillion files at startup. If you do a batch change (Add the keyword "Italy" to all 3000 of your summer holiday trip shots) the catalog program has open, modify and write back 3000 files. If you rename a file, and don't rename the sidecar file too, your meta data is no longer connected to your image. Best practice Opinion only here: Sorry. You want a unique asset tag that resides in the image. This can be an actual tag like the copyright one mentioned above, or it can be a derived tag from information in the image. This could be the EXIF time stamp (Not unique -- multiple shots per second, multiple cameras.) If your program reads makernotes, the best one is Camera model + Camera serial number + timestamp + hundredths of a second. You want a database for speed. It, of course has the unique ID You want sidecars for rebuilding your database, and for data portability. They have the unique ID. If the database crashes, it can be rebuild from the sidecars. If a sidecar is corrupted, it can be rebuilt from the database. If an image is renamed the ID can be used to reconnect it to the sidecar, and to fix the database. To make this work, you have to use a lot of timestamps. If the sidecar is more recent than the latest time stamp in the database record, then the sidecar is the authoritative record. You also have to have internal checks on data integrity. The record for an image (sidecar or database) needs a checksum to verify that that data isn't corrupt. Given the relatively fragile nature of raw files, best practice is a system that only writes zero or once to the Raw file. This is why the exif time stamp + hundredths, copyright work well. You can include the camera model and serial number in the copyright so that now the copyright message is unique to the camera. At this point you have the ability to create, and recreate a unique ID for each image. If the DAM has the ability to modify the file, you can create this ID once. This saves some time if you ever have to rebuild the database. Having as much of the metadata in the file as possible means taht it travels with the file. This is a win, but comes with the risk of potential corruption. Possibly the best strategy is to leave the original intact, and for clients who need raw data, either add metadata to a copy, or to a derived full data equivalent (e.g. DNG) Sidecars don't need to be updated in real time. The slick way to do this would be that whenever the database makes a change to a record: Make a new record that duplicates the old record in the database. Make the change in the new record. New record is flagged, "not written to sidecar" Old record is marked "obsolete" Another thread writes the sidecar files out, writing out the new one, then deleting the old one (or renaming the new one to the old one's name). Periodically you run a cleanup on the database removing obsolete records older than X days. This gives you the ability to rollback changes. This is not complete: It doesn't address the issue of non-destructive edits. Many programs now allow the creation of multiple images from the same master file, and do not create a new bitmap, but rather a file with a series of instructions for how to make the image from the master. AFAIK all such methods are proprietary. This results in a quandary as the apps that do a good job of tracking metadata may not be able to deal with the non-destructive edits. This can be critical if you crop a person out of an image, crop to emphasis a different aspect, and receive a different caption, etc. The workaround is that you always write out a new bitmap image from a serious edit. Ideally you have a script that looks for new NDEs and writes out an image based on this, copying the metadata from the master and at some point bringing it up for review for mods to the metadata. Robustness against external programs. I like having an underlying file structure organization. I like the idea that if I produce a bunch of cropped, watermarked, lower resolution, etcetera versions of an image that my catalog will track that too. But if the underlying file structure is exposed to Explorer or Finder, then you have the risk of a file being renamed or moved, and the database is no longer in sync with your file system. To budnip answers of the form "This is impossible" here's how to "Finder-proof" your image database. When an image is edited, a file system watcher notes that the file was opened. The file goes onto the 'watch' list. (the program fswatcher does this on mac. I use it to update my web page when my local copy has been edited.) When a new file appears in a monitored directory tree, it's noted. When a file is closed, this is also noted. If there has been a new file created it is checked for metadata. If the new file's metadata has a match for an existing file, then existing file metadata is used to repopulate missing data in the file. (Photoshop is notorious for not respecting all metadata.) Database is updated with the new file being marked as derivative of the original file. optionally a suffix may be added to the new file's image number, showing whether it derives directly from the original or from another derivative. To make this work, the two components are a unique id that can be calcuated from the master, and a file system monitor program that catches create, move, change, and rename events.

There are various reasons why there is a lack of good photo managers but an abundance of acceptable photo browsers: A: Programmers aren't photographers., by in large. B: In my wanderings over the internet I have found only a few people who have actually said what they are looking for. So software companies are left guessing. The article I just posted is maintained here: https://softwarerecs.stackexchange.com/questions/47756/digital-asset-management-for-photography and over the last year has tripled in length as I discuss features and tradeoffs. Readers here are free to quote from it or link to it when talking to software companies, or posting on feature request forums. C: Too many software companies are satisfied with one way to do a task, and seem to fail to realize that there have to be multiple work paths to suit different work flows. (See my article on the section on tagging. D: The big market is amateur photographers. They are interested in turning their latest snap into beautiful stuff, not with finding it among 100,000 images 10 years from now. That said: Once you have a decent gui, I don't see what the issue is. Databases while not plug and play, are mostly drone work to set up and program. The key is good user interface design. And that while hard, is a well established paradigm. Off hand I would put the difficulty of making a good photo manager as being far easier than writing even a single clever module for an image processor.

I've found and read the bkwine article several times. The author has found a 'least bad' solution, but not a good solution. At this point in time there is not a DAM on the market that meets all the needs of a professional photographer. I've been thinking a lot about photo management. I'm starting to avoid the word 'DAM' as it increasingly refers to industrial sized software costing tens or hundreds of thousands of dollars. So let's look at what I mean by a photo manager: Browser Pix initially come in in bunches, and as such they go somewhere in some folder structure on your computer. Many people will use some combination of Year/Month/ and string to describe event. Often remember that the shot took place on your trip to Italy, or that it was the Smith & Brown wedding is sufficient. Tagger But what do you do when you are looking for the closeup of a butterfly. It was an incidental pic on some holiday, but which one. Now metadata comes into play. If all your holiday shots are tagged 'Holiday' and your program can search existing metadata, your problem is solved. Search for holiday and focal distance less than 5 feet. You still may have a bunch to wade through. If, in addition to some general keywords for the batch you add a few per image you have a big win. E.g. "Butterfly" Tagging is hard. You want to tag it with multiple things. E.g: Describe the scene. Identify the location. GPS is fine, but "Lock Ness, Scotland" or Kensington market, Toronto, Ontario, Canada is easier to visualize. ID the people in the scene. Classify them more generally. (Woman with child; Young boy...) Describe the technical aspects -- close up, high/low key, lighting One or more classes of description about the scene -- weather, mood. These are facets. I prefer to go through a set of images several times, concentrating on one of these at a time. Sometimes a facet is irrelevant. Weather makes no sense for an interior shot. If you do facets, you need a way to search for images that don't have an entry for facet X. You also need a way to mark a facet as irrelevant. Crudely you can implement these with constructs like WEA:Cloudy but then you still have to be able to search for images that don't have WEA:* as a keyword. And you have to decide on what to do where it's not relevant. WEA:N/A Having some kind of support for actual facets would be a big win. Searcher No point in tagging if you can't search the data. Two programs I trialed, Mylio and Photoshop Supreme, had no provision to search exif data -- where the stuff like time of day, and focal length, and camera model is kept. One program allows you to search for only one tag at a time. I can search for Holiday. Or I can search for butterfly. But I can't search for shots that have both "Holiday" and "butterfly" Ideally you want full boolean search support with and, or, not, parentheses for grouping, and wild cards for partial matches. Version tracker A photograph for a professional may have a long history. You often have a shot, then export it in some altered form (cropped, resized, sharpened, colour adjusted, watermarked) Nice to be able to find the original 5 years from now. One recommended practice I ran into had the following: Master image was Raw. Archive version was digital negative. Processing version was 16 bit tiff or PSD Delivered version was tiff or jpeg. This requires a minimum of 4 versions. Add to that: * Watermarked versions. * Reduced resolution versions for web pages. * Colour matched versions for specific printing environments. * Cropped versions for mobile web pages. So that's the base case. Implementations differ, and they refine this somewhat. Online resources Impulse Adventure (site: https://www.impulseadventure.com/photo/) Unfortunately out of date. But still several good articles. Catalogs and Multiple versions. https://www.impulseadventure.com/photo/flow-catalog-versions.html Important Features of Catalog Software. https://www.impulseadventure.com/photo/flow-catalog-features.html Controlled Vocabulary (site: https://www.controlledvocabulary.com/ ) Using Image Databases to Organize Image Collections http://www.controlledvocabulary.com/imagedatabases/ Also has a good forum/mailing list. Requirements: Server requirments I can see implementing this in one of two ways: Either as a stand alone program or as a local web server. The latter has the advantage that it would scale for family or small photo business. Cloud services are slow when you are talking about 10-12 Mbyte files. My network connect takes several seconds per MByte. Cloud services for metadata have to be well optimized -- you really don't want to issue 3000 keyword change requests individually when you change the spelling of a keyword. So: Not cloud based. Runs on Mac or on local apache web server. Keyword handling Fast keywording. Aperture allows drag and drop from a list, multiple sets of hotkeys for words used frequently, copy paste of keywords from one photo to another, and keywords organized in folders. It also allows search for a keyword, and a list matching what you typed so far appears. Other programs that have good keywording include IMatch and Photomechanic. One of the key aspects of this is to have multiple ways to do things. Full access to standard metadata: EXIF, ITPC, subject to limits of the file format. Controlled vocabulary. I want an extra step to add a new keyword to my list of keywords. This helps with the the Sommer Vacashun problem. Hierarchial vocabulary. E.g. Separate entries for Birds -> raptors -> falcon and Planes -> fighters -> falcon. Parents are stored with keywords. Moving a keyword in the master list, or changing spelling, corrects all usage in photos. This can be done as a background task. Parent items are automatically entered as keywords. (With the correct database linkage, this comes free as a side effect of the point above. Synonyms -- I can define "Picea glauca" as a synonmym for "White Spruce" entering one, enters the other. Facets: For a set of pictures I want to be able to define a set of facets or categories for collections or folders. Facets would be things like: Weather; Who; Where; Ecosystem; Season; Lighting Not all collections would have all facets, but a collection having a facet would nag me to put it in. A facet would have a negation for not applicable (Weather isn't applicable inside a house; Who isn't applicable in a landscape shot.) Facets allow me to go through a collection in multiple passes and get the missing keywords. Searching Complex searches: Find all shots between 2012 and 2015 shot in December or January, shot with my Nikon D70, with keyword "snow" rating of 3 or better shot after 3pm in the day. (Yes, I do use searches like that) Saved Searches. These are the equivalent of smart albums in Aperture. As new pix meet the standards they would be shown. Version Tracking Version tracking If a lower resolution, cropped, photoshopped, composited or a black and white image is produced from a master, the system should show that it's a derived image, and allow access to the master. A master should be able to list derived images. Derived images are not linear but form a multi-branched tree. If my camera produces JPEG and Raw versions, I want the JPEG to be shown as being derived from the Raw version. Metadata applied to a master should propagate down to derived images. Some form of exception handling for this: e.g. -keyword to prevent a people identifier being applied to an image where that person was cropped out. Ability to track through external editing programs. E.g. If I edit a program in photoshop, it will mark the PSD file as being derived, restore as much of the metadata as the PSD format allows. If Photoshop is used to create a jpeg image, that too is tracked. Data robustness All metadata is indexed. Metadata is also written to sidecar files. Where possible metadata is written to the image file itself. (optional -- can stress automated backup systems) Through file system watching, name changes and directory reorganization are caught. Relevant sidecars are also renamed, and the database updated with new file location/name. Sidecar contents include the name of their master file. Should be possible to rebuild entire database from images + sidecars. Should be able to restore all file metadata from database. This requires a lot of under-the-hood time stamps to determine which has priority. All database actions should be logged and journaled, so they are reversible. Reasonable speed with catalogs of more than 100,000 images. Support for previews of all common image formats and most raw formats. Previews and thumbnails are treated as versions of the master. They inherit metadata. Nice to have: Simple non-destructive editing -- crop, brightness, contrast. Rating system Smart albums Drag and drop functionality with other mac apps. Suggestions? Notes on current state of the art: Nothing I've found supports version tracking, especially through an external program. Lightroom and Aperture both support simple versions -- different edits on same master. Aperture supports Stacks -- a group of related pictures. Lightroom: Doesn't support PNG, very clunky interface, slow on large catalogs; Mylio home version doesn't support hierarchical keywords; doesn't index exif information, does not allow or syntax for searches, Photomechanic is fast for keywording and culling, but has very limited search capability. IMatch. Possible contender, Requires MS windows box. Photo Supreme: Erratic quirks. Crashes. One man shop. Can't search Exif in useful way. Fotostation: AFAIK no underlying database. Has to read metadata from images/sidecar files on startup. Slow after 10K images. (They have server based software too that is big bucks.) Luminar: A DAM has been promised Real Soon Now, but no demos, storyboards or feature lists have been published. There is a claim that it is in beta, but no one on their fairly active forum will admit to being part of the beta group. Affinity: Similar to Luminar. Commandline tools Much of the special features for version tracking could be implemented with scripts using calls to these programs. ImageMagick -- good for whole-image conversions, also can read/write internal metadata and sidecars. Exiftool -- read/write exif data reads most makernotes. fswatch -- not really an image processor, but hooks into the operating system and can alert when files have changed -- modified, renamed, moved. Enterprise level WebDAM No real information about capabilities on web site. Extensis. Expensive. Bynder. Joke program. Cloud based set of shoeboxes. WIDEN. Cloud only. Asset Bank. Starts at $500/month for up to 50 users. Metadata Storage There are three places metadata can be stored: In the image. In a database. In a separate file for each image (sidecar file) Typically these files have the same name as the primary file, but a different suffix. If at least some cataloging information is written to the image, then you can reconnect a file to your database. In principle this can be a single unique ID. This saves you from: You moved or renamed an image file. If you can write more info into the file -- keywords, captions -- then you are saved from: Your database is corrupted. You upgraded your computer and your database program doesn't work there. Sidecar files allow you to recover all your metadata if your database crashes. Downsides of storing data in the image Writing to the original files can corrupt the file. Most RAW formats are well understood enough now to at least identify and replace strings of metadata with the same length string. If you tell your camera to put the copyright string Copyright 2018 J. Random Shutterbug Image XXXX-XXXX-XXXX-XXXX-XXXX-XXXX Then as long as the DAM keeps that string the same length you are golden. Keeping all metadata (or as much as you can) in the original images makes for very slow access. Your program has to read at least the first few blocks of every image. Depending on the file structure, adding too much data may require rewriting the entire file. Any program that moves the boundaries of data sub blocks better be well tested. Writing data back is time consuming. Some file formats don't have any metadata capability. Some file formats (Photoshop PSD) are noted for mangling metadata. A glitch during the write process can corrupt the image file. The alternative, writing a new file, then replacing the old file requires that the entire file be both read and written, rather than just a chunk of it. This has serious performance issues. Downsides of Databases Databases are fast, but they are blobby, and you are writing into the middle of blobs of data. If the implementation of the database is solid, there isn't much to worry about. But hard disks have errors, and a single error can make a database partially or fully unusable. Good database design has redundancy built in so that you can repair/rebuild. Databases are frequently proprietary. Data may be compressed for speed. Getting your data out may be tricky. (Problem for people using Apple Aperture) Databases frequently are optimized in different ways. In general robustness is gained at the cost of performance and complexity. One compromise is to write all changes first to a transaction file (fast...) and then a background process does the database update in the background. This slows down access some: Have to check both the main database and the transaction file, but unless the transaction file gets to be bigger than memory, this shouldn't be noticeable. Downsides of Sidecars You have to read a zillion files at startup. If you do a batch change (Add the keyword "Italy" to all 3000 of your summer holiday trip shots) the catalog program has open, modify and write back 3000 files. If you rename a file, and don't rename the sidecar file too, your meta data is no longer connected to your image. Best practice Opinion only here: Sorry. You want a unique asset tag that resides in the image. This can be an actual tag like the copyright one mentioned above, or it can be a derived tag from information in the image. This could be the EXIF time stamp (Not unique -- multiple shots per second, multiple cameras.) If your program reads makernotes, the best one is Camera model + Camera serial number + timestamp + hundredths of a second. You want a database for speed. It, of course has the unique ID You want sidecars for rebuilding your database, and for data portability. They have the unique ID. If the database crashes, it can be rebuild from the sidecars. If a sidecar is corrupted, it can be rebuilt from the database. If an image is renamed the ID can be used to reconnect it to the sidecar, and to fix the database. To make this work, you have to use a lot of timestamps. If the sidecar is more recent than the latest time stamp in the database record, then the sidecar is the authoritative record. You also have to have internal checks on data integrity. The record for an image (sidecar or database) needs a checksum to verify that that data isn't corrupt. Given the relatively fragile nature of raw files, best practice is a system that only writes zero or once to the Raw file. This is why the exif time stamp + hundredths, copyright work well. You can include the camera model and serial number in the copyright so that now the copyright message is unique to the camera. At this point you have the ability to create, and recreate a unique ID for each image. If the DAM has the ability to modify the file, you can create this ID once. This saves some time if you ever have to rebuild the database. Having as much of the metadata in the file as possible means taht it travels with the file. This is a win, but comes with the risk of potential corruption. Possibly the best strategy is to leave the original intact, and for clients who need raw data, either add metadata to a copy, or to a derived full data equivalent (e.g. DNG) Sidecars don't need to be updated in real time. The slick way to do this would be that whenever the database makes a change to a record: Make a new record that duplicates the old record in the database. Make the change in the new record. New record is flagged, "not written to sidecar" Old record is marked "obsolete" Another thread writes the sidecar files out, writing out the new one, then deleting the old one (or renaming the new one to the old one's name). Periodically you run a cleanup on the database removing obsolete records older than X days. This gives you the ability to rollback changes. This is not complete: It doesn't address the issue of non-destructive edits. Many programs now allow the creation of multiple images from the same master file, and do not create a new bitmap, but rather a file with a series of instructions for how to make the image from the master. AFAIK all such methods are proprietary. This results in a quandary as the apps that do a good job of tracking metadata may not be able to deal with the non-destructive edits. This can be critical if you crop a person out of an image, crop to emphasis a different aspect, and receive a different caption, etc. The workaround is that you always write out a new bitmap image from a serious edit. Ideally you have a script that looks for new NDEs and writes out an image based on this, copying the metadata from the master and at some point bringing it up for review for mods to the metadata. Robustness against external programs. I like having an underlying file structure organization. I like the idea that if I produce a bunch of cropped, watermarked, lower resolution, etcetera versions of an image that my catalog will track that too. But if the underlying file structure is exposed to Explorer or Finder, then you have the risk of a file being renamed or moved, and the database is no longer in sync with your file system. To budnip answers of the form "This is impossible" here's how to "Finder-proof" your image database. When an image is edited, a file system watcher notes that the file was opened. The file goes onto the 'watch' list. (the program fswatcher does this on mac. I use it to update my web page when my local copy has been edited.) When a new file appears in a monitored directory tree, it's noted. When a file is closed, this is also noted. If there has been a new file created it is checked for metadata. If the new file's metadata has a match for an existing file, then existing file metadata is used to repopulate missing data in the file. (Photoshop is notorious for not respecting all metadata.) Database is updated with the new file being marked as derivative of the original file. optionally a suffix may be added to the new file's image number, showing whether it derives directly from the original or from another derivative. To make this work, the two components are a unique id that can be calcuated from the master, and a file system monitor program that catches create, move, change, and rename events.

@EddCh Thanks for the lead on Alien Skin X3. I followed your link, and read the transcript of the vid. As a file manager/browser it may be acceptable. It's not clear however that you can do any of the following tasks: * Rename a keyword, and have that change propagate to all images that use that keyword. * Add a keyword to multiple photos at once. * Save a search as a virtual folder (smart album) * Move or rename a file is a program outside X3 and have the program keep track of it. * Track the file through an external editor, and keep metadata in sync with the external editor native format. * Support Controlled vocabulary. * Support hierarchical vocabulary.

There are NO worthy DAMs for mac right now: =iMatch, Dammion windows only. =Digikam -- very clunky on mac due to importing an entire windowing system. Limited. =Photosupreme. Wretched search. Unable to use exif metadata for searches. This is my wishlist, posted on another site. Currently using Apple Aperture. Need a replacement. Requirements: Server requirments Not cloud based. Runs on Mac or on local apache web server. Keyword handling Fast keywording. Aperture allows drag and drop from a list, multiple sets of hotkeys for words used frequently, copy paste of keywords from one photo to another, and keywords organized in folders. Other programs that have good keywording include IMatch and Photomechanic. Full access to standard metadata: EXIF, ITPC, subject to limits of the file format. (Additional fields are written to sidecars) Controlled vocabulary. I want an extra step to add a new keyword to my list of keywords. Hierarchial vocabulary. E.g. Separate entries for Birds -> raptors -> falcon and Planes -> fighters -> falcon. Parents are stored with keywords. Moving a keyword in the master list, or changing spelling, corrects all usage in photos. This can be done as a background task. Parent items are automatically entered as keywords. (With the correct database linkage, this comes free as a side effect of the point above. Synonyms -- I can define "Picea glauca" as a synonmym for "White Spruce" entering one, enters the other. Facets: For a set of pictures I want to be able to define a set of facets or categories for collections or folders. Facets would be things like: Weather; Who; Where; Ecosystem; Season; Lighting Not all collections would have all facets, but a collection having a facet would nag me to put it in. A facet would have a negation for not applicable (Weather isn't applicable inside a house; Who isn't applicable in a landscape shot.) Facets allow me to go through a collection in multiple passes and get the missing keywords. Searching Complex searches: Find all shots between 2012 and 2015 shot in December or January, shot with my Nikon D70, with keyword "snow" rating of 3 or better shot after 3pm in the day. Saved Searches. These are the equivalent of smart albums in Aperture. As new pix meet the standards they would be shown. Version Tracking Version tracking: If a lower resolution, cropped, photoshopped, composited or a black and white image is produced from a master, the system should show that it's a derived image, and allow access to the master. A master should be able to list derived images. Derived images are not linear but form a multi-branched tree. If my camera produces JPEG and Raw versions, I want the JPEG to be shown as being derived from the Raw version. Metadata applied to a master should propagate down to derived images by default. Some form of exception handling for this: e.g. -keyword to prevent a people identifier being applied to an image where that person was cropped out. Ability to track through external editing programs. E.g. If I edit a program in photoshop, it will mark the PSD file as being derived, restore as much of the metadata as the PSD format allows. If Photoshop is used to create a jpeg image, that too is tracked. Data robustness All metadata is indexed. Metadata is also written to sidecar files. Where possible metadata is written to the image file itself. (optional -- can stress automated backup systems) Through file system watching, name changes and directory reorganization are caught. Relevant sidecars are also renamed, and the database updated with new file location/name. Sidecar contents include the name of their master file. Should be possible to rebuild entire database from images + sidecars. Should be able to restore all file metadata from database. This requires a lot of under-the-hood time stamps to determine which has priority. All database actions should be logged and journaled, so they are reversible. Reasonable speed with catalogs of more than 100,000 images. Support for previews of all common image formats and most raw formats. Previews and thumbnails are treated as versions of the master. They inherit metadata. Nice to have: Simple non-destructive editing -- crop, brightness, contrast. Rating system Smart albums Drag and drop functionality with other mac apps. Suggestions? So far: Nothing I've found supports version tracking, especially through an external program. Lightroom and Aperture both support simple versions -- different edits on same master. Aperture supports Stacks -- a group of related pictures. Lightroom: Doesn't support PNG, very clunky interface, slow on large catalogs; Mylio home version doesn't support hierarchical keywords; doesn't index exif information, does not allow or syntax for searches, Photomechanic is fast for keywording and culling, but has very limited search capability. IMatch. Possible contender, Requires Ms windows box. Photo supreme: Erratic quirks. Crashes. One man shop. Can't search Exif in useful way. Fotostation: AFAIK no underlying database. Has to read metadata from images/sidecar files on startup. Slow after 10K images. (They have server based software too that is big bucks.) Enterprise level * WebDAM No real information about capabilities on web site. * Extensis. Expensive. * Bynder. Joke program. Cloud based set of shoeboxes. * WIDEN. Cloud only. * Asset Bank. Starts at $500/month for up to 50 users. To budnip answers of the form "This is impossible" here's how version control could be implemented: For each master image generate a unique ID based on the content of the file. This could be a checksum of the file preview image, or Camera model+serial number + shutter count. The latter is preferred as it can be regenerated. In some cases previews can be modified which changes the checksum This ID is written to a set of fields in meta data that most editors will leave at least one intact. If the master is unwritable, it's written to a sidecar file. The ITPC field "Title" is designed for this, despite it's odd name. In addition all metadata in the file is slurped into a database. When an image is edited, a file system watcher notes that the file was opened. The file goes onto the 'watch' list. When a new file appears in a monitored directory tree, it's noted. When a file is closed, this is also noted. If there has been a new file created it is checked for metadata. If the new file's metadata has a match for an existing file, then existing file metadata is used to repopulate missing data in the file. (Photoshop is notorious for not respecting all metadata.) Database is updated with the new file being derivative of the original file. optionally a suffix may be added to the new file's image number, showing whether it derives directly from the original or from another derivative.

My feature requests for DAM * Runs on Mac or on local apache web server. * Fast keywording * Full access to standard metadata, subject to limits of the file format. (Additional fields are written to sidecars) * Controlled vocabulary * Hierarchial vocabulary. E.g. Separate entries for Birds -> raptors -> falcon and Planes -> fighters -> falcon. Parents are stored with keywords. * Parent items are automatically entered as keywords. (With the correct database link, this comes free as a side effect of the point above. * Synonyms -- I can define "Picea glauca" as a synonmym for "White Spruce" entering one, enters the other. * Version tracking If a lower resolution or a black and white image is produced from a master, the system should show that it's a derived image, and allow access to the master. A master should be able to list derived images. Derived images are not linear but form a multi-branched tree. * Metadata applied to a master should propagate down to derived images. * Ability to track through external editing programs. E.g. If I edit a program in photoshop, it will mark the PSD file as being derived, restore as much of the metadata as the PSD format allows. If Photoshop is used to create a jpeg image, that too is tracked. * All metadata is indexed. * Metadata is also written to sidecar files. * Through file system watching, name changes and directory reorganization are caught. Relevant sidecars are also renamed, and the database updated with new file location/name. * Support for previews of all common image formats and most raw formats. * Reasonable speed with catalogs of more than 100,000 images. * Ability to rebuild corrupt database from sidecars. Nice to have: * Simple non-destructive editing -- crop, brightness, contrast. * Rating system * Smart albums Suggestions? So far: * Nothing I've found supports version tracking. * Lightroom: Doesn't support PNG, very clunky interface, slow on large catalogs; * Photomechanic is fast for keywording and culling, but has very limited search capability. * Photo supreme: Erratic quirks. Crashes. One man shop. * Fotostation: AFAIK no underlying database. Has to read metadata from images/sidecar files on startup. Slow after 10K images. (They have server based software too that is big bucks.) Similar question: https://softwarerecs.stackexchange.com/questions/37348/advanced-software-to-organize-and-manage-a-life-of-photos That question has a more general set of requirements and received no answers. To budnip answers of the form "This is impossible" here's how it could be implemented: * For each master image generate a unique ID based on the content of the file. This could be a checksum of the file preview image, or Camera model+serial number + shutter count. * This ID is written to a set of fields in meta data that most editors will leave at least one intact. If the master is unwritable, it's written to a sidecar file. * I addition all metadata in the file is slurped into a database. * When a file is edited, a file system watcher notes that the file was opened. The file goes onto the 'watch' list. * When a file is closed, this is also noted. If there has been a new file created it is checked for metadata. If the new file's metadata has a match for an existing file, then existing file metadata is used to repopulate missing data in the file. (Photoshop is notorious for not respecting all metadata.) * Database is updated with the new file being derivative of the original file. * optionally a suffix may be added to the new file's image number, showing whether it derives directly from the original or from another derivative.

What I hope your DAM does. Runs on Mac or on local apache web server. Fast keywording Full access to standard metadata, subject to limits of the file format. (Additional fields are written to sidecars) Controlled vocabulary Hierarchial vocabulary. E.g. Separate entries for Birds -> raptors -> falcon and Planes -> fighters -> falcon. Parents are stored with keywords. Parent items are automatically entered as keywords. (With the correct database link, this comes free as a side effect of the point above. Synonyms -- I can define "Picea glauca" as a synonmym for "White Spruce" entering one, enters the other. Version tracking If a lower resolution or a black and white image is produced from a master, the system should show that it's a derived image, and allow access to the master. A master should be able to list derived images. Derived images are not linear but form a multi-branched tree. Metadata applied to a master should propagate down to derived images. Ability to track through external editing programs. E.g. If I edit a program in photoshop, it will mark the PSD file as being derived, restore as much of the metadata as the PSD format allows. If Photoshop is used to create a jpeg image, that too is tracked. All metadata is indexed. Metadata is also written to sidecar files. Through file system watching, name changes and directory reorganization are caught. Relevant sidecars are also renamed, and the database updated with new file location/name. Support for previews of all common image formats and most raw formats. Reasonable speed with catalogs of more than 100,000 images. Ability to rebuild corrupt database from sidecars. Nice to have: Simple non-destructive editing -- crop, brightness, contrast. Rating system Smart albums Suggestions? So far: Nothing I've found supports version tracking. Lightroom: Doesn't support PNG, very clunky interface, slow on large catalogs; Photomechanic is fast for keywording and culling, but has very limited search capability. Photo supreme: Erratic quirks. Crashes. One man shop. Fotostation: AFAIK no underlying database. Has to read metadata from images/sidecar files on startup. Slow after 10K images. (They have server based software too that is big bucks.) Similar question: Advanced software to organize and manage a life of photos That question has a more general set of requirements and received no answers. To budnip answers of the form "This is impossible" here's how it could be implemented: For each master image generate a unique ID based on the content of the file. This could be a checksum of the file preview image, or Camera model+serial number + shutter count. This ID is written to a set of fields in meta data that most editors will leave at least one intact. If the master is unwritable, it's written to a sidecar file. I addition all metadata in the file is slurped into a database. When a file is edited, a file system watcher notes that the file was opened. The file goes onto the 'watch' list. When a file is closed, this is also noted. If there has been a new file created it is checked for metadata. If the new file's metadata has a match for an existing file, then existing file metadata is used to repopulate missing data in the file. (Photoshop is notorious for not respecting all metadata.) Database is updated with the new file being derivative of the original file. optionally a suffix may be added to the new file's image number, showing whether it derives directly from the original or from another derivative