Artificial Intelligence and Machine Learning focus on creating systems that build adaptive models from available data.
Data ingestion pipelines and governance processes for maintaining these systems in production are key aspects of a successful enterprise AI strategy.
Artificial Intelligence and Machine Learning is now widespread in many areas of business and everyday life. The availability of ever greater compute capabilities at lower costs coupled with advances in neural network architectures has enabled the deployment of so called 'intelligent' solutions within more and more systems and processes at cost .
While the impact of AI and ML on business is evident, the strategy to integrate solutions into exisiting workflows and the associated cost of maintaining these AI enhanced workflows in enterprise production situations is often underestimated
Quality and Availability of Data is another often overlooked aspect of the AI ML production pipeline. The accuracy of many sophisticated Neural Network models depends on available, accurate data in readily ingestible formats both for training the models and for efficient execution in production situations. The data ingestion and processing phase is arguably the key factor in the success of an AI enhanced business process.
It follows that a successful AI strategy considers the full pipeline of AI enhanced process development from the inital case studies to determine optimal algorithms and where AI technologies are best deployed, to data identification and ingestion strategy including processing and cleansing aspects, deployment and execution of the algorithms within the wider business process and governance cycles covering maintenance, upgrades migration and other common aspects of maintaining enterprise systems in production.
- Model must be adapted to application
The focus should always be a results oriented approach, the basis of AI/ML is building models from input / output data and refining these to improve outcomes. The ‘model’ is a direct reflection of the required outcomes.
- What are the key business / functional requirements for the model, i.e what is actually required by the application. The aim should always be optimisation of resources, efficiency both in terms of cost and compute assets is a key tradeoff requirement.
Common requirements accross most IT infrastructure architectures and enterprise systems