Return on Investment calculations/tracking key performance indicators - Museum Technology Projects.
While there is no pre-existing information to fully answer your question, we've used the available data to pull together key findings: Museums, like any business, regularly conduct return-on-investment (ROI) analyses on proposed projects and track the key performance indicators (KPIs) in their business. However, the diversity of possible technology projects combined with the different circumstances faced by each museum mean that there are no generalized rules-of-thumb, and individual museums do not publish the details their internal operations.
Below you'll find an outline of our research methodology to better understand why information you've requested is publicly unavailable, as well as a deep dive into our findings.
The term "museum technology project" is extremely broad in scope. To guide our search, we used the Ontario Museum Association (OMA) Tipsheet for Museum Technology Projects. This defines technology projects as ranging from technology implementations like interactive panels in galleries, to outreach and education solutions, to the digitization of collections, to web-based enhancements. Therefore, for purposes of this report, we understand the term to mean practically any technological upgrade of any aspect of the museum's exhibits and/or experience.
We first reviewed the annual financial statements of several museums. We found that art museums, like the Portland Art Museum, tended not to have a line item for technology (or any related term) at all. Science museums like the American Museum of Natural History, on the other hand, do line-item their technology usage; e.g., the AMNH, with an operating expense budget of $175 million, spent $8.2 million on communications and digital media and $7.8 million on information technology in 2017. However, none of the financial reports that we studied contain a detailed explanation as to the museum's ROI calculations in regard to its technology projects. We hypothesize that this is because museums, just like any other business, have little motivation to publicize their internal discussions on the matter.
As we are reliant on these public sources in our research, we are unable to provide hard numerical data to fully answer the question. However, we can provide some general answers which should make it clear why more information is not available.
DO MUSEUMS CALCULATE ROI ON TECHNOLOGY PROJECTS?
The OMA Tipsheet itself is evidence that museums do an ROI calculation when considering projects, though it does not provide any information on how that calculation would be performed. Other sources concur: Martijn Pronk, the Head of Digital Communication at the Van Gogh Museum, states that in modern museums, "digital is still a layer on top of real life. It is not fully integrated. In my opinion this is (among other factors) a result of the organizational structure of museums. Management hesitates to commit to projects they cannot fully comprehend, the return on investment is absent or unclear, the digital mandate suffers from stakeholders with opposing demands, et cetera." In other words, being very conservative as a group, museum management will perform a traditional ROI on all technological projects within the museum: Subtract the cost from the gain and divide that by the cost.
We hypothesize that this is why science museums are more likely to have line items for technology investments than art museums (as explained under Methodology): It is simply easier to calculate and demonstrate the ROI on a technology project for a science museum (e.g., the expected gain from more public interest).
GAINS FROM TECHNOLOGY PROJECTS
While there is abundant evidence that museums perform a traditional ROI analysis on any proposed technology project, it is telling that even companies who wish to convince museums of the benefits of such projects do not attempt to advertise either costs or monetary benefits. In fact, after an extensive search, we were unable to find a publicly published case study in which an ROI was presented. We judge that this is because the wide range of possible technology projects and the diverse nature of museums means that there are too many factors for anyone to develop universal rules-of-thumb. To illustrate this, let us consider two very different projects.
The first comes from a recent white paper from Proximity Studio focuses on the benefits of its Bluetooth "Beacon" technology. Traditional printed labels offer only limited space to describe the exhibit, are time-consuming to update, and are not crowd-friendly. Printed guidebooks are costly, wasteful, and limit your language options. An audio guide with a headset is expensive to implement in the first place, but moreover is expensive to maintain as the technology becomes obsolete. Proximity touts its "Beacon" technology as a way of enhancing the visitor experience at a low cost compared to those older methods, while collecting valuable KPI data for the museum (see below).
Second, many technology projects have ROIs which are difficult to translate into dollars. For example, "mitigating the risk of losing valuable information and records ... due to decay and passage of time" is the motivation behind the current collaboration between Digital Divide Data, Amazon Web Service, Intel, and the National Museums of Kenya "to digitize one of the largest collections of Archeology and Paleontology in the world." Digitization enables the public to interact with virtual or 3-D replicas of irreplaceable cultural artifacts, enhancing the visitor experience while protecting and preserving the original.
Obviously, these two projects would have to calculate and present their ROIs very differently, and in the latter case, the real ROI would be scientific rather than financial.
TRACKING KEY PERFORMANCE INDICATORS
Just as museums run ROI analysis on their technology programs, it is also clear that they track key performance indicators (KPIs) in general. For example, the Science Museum Group Plan for 2013–14 (which was publicly published in June 2017, and so represents the most up-to-date information available), lays out 14 distinct KPIs, including "virtual visit numbers" (online visits) for tracking the success of its programs. In fact, this topic is explored in detail in the book, "Measuring Museum Impact and Performance: Theory and Practice," by John W. Jacobson. (An overview of Jacobson's views, particularly on developing universal standards for measuring museum KPIs, was recently published by the American Alliance of Museums.)
Despite an extensive search, we were unable to find a public source that discussed the best KPIs to monitor the success of a museum's technology projects as a separate subject. As in the case of calculating an ROI, we hypothesize that it would be impossible to create a one-size-fits-all KPI list and criteria between two different technology projects in two different museums.
As an interesting aside, we found in the course of our research that technology firms are touting their products as a means of tracking the museum's KPIs. For example, a white paper from Proximity Studio regarding their "Beacons" technology solution for museums states, "While your visitors are traveling between the rooms, a museum app can be leveraged to collect data and discover what parts of your museum are particularly popular—or unpopular. This kind of highly specific insight will let managers make better staffing decisions and could lead to better-planned exhibits and campaigns."
While there is no doubt that museums regularly perform ROI and KPI analyses on proposed technology projects, we were unable to locate specific ROI and KPI formulas or practices. We hypothesize that these metrics are so heavily dependent on the specific circumstances of the museum and details of the project that it is impossible for anyone to provide valid rules-of-thumb, and that the museums themselves see no benefit in publishing their internal policies and analyses.