Blog

Capitalization and Re-Use of A.I.

Artificial Intelligence and Machine Learning are accelerating at a rapid pace, spurred on by innovations in hardware and the availability of data. Businesses are constantly trying to catch up and drink this elixir to generate profits and/or increase savings. However, many times the cure can be worse than the malady.

In this case, by investing in Artificial Intelligence solutions unwisely businesses run the risk of losing revenue and spending countless money on wasted opportunities. Some of the main reasons for this can be creating grand machine learning projects that have no practical business use, spending time gathering data that is no longer relevant or properly suited for the applications, investing in a data science team without proper data engineering support, developing a poor infrastructure which leads to months, if not years, of model training and tweaking for minimal returns, consistently starting from scratch for any new machine learning project, and letting resources stay idle in the cloud when there are lulls in the development process as well as poorly estimating costs of investing in a machine learning project.

The solutions are quite simple, really, though it takes a disciplined business to adhere to basic guidelines:

1. Customers First – A business should always orient themselves to address customer needs and understand what the customer truly wants. Some may argue that the customer does not always know what they truly want. Henry Ford once famously quoted: “If I had asked them for what they want, they would have asked for a faster horse”. Yet, the two ideas are not exactly inconsistent. Businesses and corporations should not strictly adhere to what a customer says they want, but they should understand a customer’s mindset, fears and concerns as well as their desire to stay relevant in an increasingly crowded marketplace. Machine Learning solutions can pose a unique advantage as a disruptor in many industries, but the products must be designed to address the basic customer needs and motivations first.

2. Develop Reusable Components – Divide out machine learning projects into individual components that are reusable. Create a feature store. Many companies today are developing their own feature store, thus obviating the need to re-develop features. Data scientists do not need to spend their time developing features that were already created several projects ago, and they can thus spend time ideating new ML products.

3. Compartmentalization of Tasks – While it is true that a data scientist should be a good engineer and be capable of producing production-worthy code, they should not be asked to spend their time productionizing their models. They should spend their time instead pushing the boundaries of current machine learning methods and finding ways to leverage the data to produce new insights. A data engineering team should fill the gap and provide support for the data scientists by spending time to optimize the pipeline for delivery of the resources that data scientists need. ETL jobs should be optimized as well as data centers should be properly localized so that data is made more readily available for ingestion, analysis and experimentation by data scientists. Machine learning proofs of concept should be productionized by data engineers who are adept at coding and fulfilling the infrastructure requirements necessary to maximize utility of the models.

4. Develop Data Science Standards – Data scientists need a proper way to benchmark their models. Without a benchmark, data scientists may end up endlessly iterating over their models, never satisfied with the performance of their models. This not only wastes their time, it wastes the time and money of companies that have invested in these projects. Machine learning and data science projects need to have proper expectations set before the initiation of any new venture to know when their outcome is “good enough”. This may include leveraging open-source machine learning models, adapting and integrating third-party automated ML solutions such as H20 and Auto ML, or indulging in the necessary research beforehand of what the current state of the art model is and their performance in the real world.

5. Assess Market Potential – Prior to initiating a new machine learning project, businesses and corporations should first understand the revenue-generating potential of a new ML product. Proper analysis of the marketability of a new ML product will help businesses identify boundaries within which to properly budget their ML ventures and identify the scenarios in which they need to abandon the venture or suspend it if costs and resources exceed a certain threshold.

6. When Idle, Tighten Budget – Diligence towards optimizing the use of resources should be a priority when investing in any project, especially a machine learning project. Leveraging cloud solutions is appropriate when resources are in high demand and personal computers are not sufficient to perform the task. However, sometimes it is cheaper to invest in servers with large computing resources than it is to leverage cloud solutions such as Amazon AWS and Microsoft Azure, and a business must properly assess the computing costs and requirements to make this decision.

These are some key guidelines that can help constrain expenditures and optimize processes for businesses to properly invest in machine learning projects. There are many more ideas, however, we believe these are central to creating successful revenue-generating machine learning products.