AI Apps: Driving innovation from development to production

~ satonaoki

Apps on Azure Blog > AI Apps: Driving innovation from development to production

https://techcommunity.microsoft.com/t5/apps-on-azure-blog/ai-apps-driving-innovation-from-development-to-production/ba-p/4146614

AI is driving a generational shift in applications, shaping and redefining every application, while forming the core of a new, previously impossible, class of intelligent applications. The expectations for AI applications continue to grow more ambitious: IDC forecasts a billion new logical applications to be built by 2028[1]. Increasingly AI is playing a defining role in these new applications. App modernization, already a top business priority, is accelerating for 69% of businesses to better take advantage of and support AI initiatives[2]. These trends are well underway with 92% of developers already using AI[3] as part of their development lifecycle and toolchain.


 


Taken as a whole we see the true impact of AI, driving new value and innovation into core business applications while simultaneously accelerating the process of building and modernizing those same applications and making the entire end to end application lifecycle faster and more efficient.


 


Delivering Impact with AI Apps


Organizations of all sizes are already redefining core business processes with intelligent applications built on generative AI models and massive volumes of business data, driving increased revenue, refining business operations, making customer interactions more personal and more engaging while creating a faster path to innovation. These leading businesses are showing the market what’s possible:



  • BMW is using Azure to operate the largest over-the-air (OTA) updatable vehicle fleet in the world, serving 13M+ active owners with their vehicles globally on their My BMW and MINI apps.

  • The NBA has achieved a +900% YoY increase in referee performance conversations on their Azure AI-powered REPS (Referee Engagement and Performance System) app in the first year. The REPS app aida referees and management in evaluation, collaboration, training and development to improve on-court performance.

  • Canary Speech, a biotechnology startup, uses the power of Azure AI with Azure’s application platform to build breakthrough biomarker technology that analyzes conversational speech in real time to assess and monitor for early disease detection.


These stories illustrate the undeniable impact of intelligent applications, even in the early days of the market. This week at Microsoft Build, we’re moving closer to helping every business achieve equally ambitious results by tackling the top opportunities we’ve heard from our customers:



  • Making it easier for every developer to build fast with the skills they have and the tools they love

  • Removing friction across the entire app lifecycle

  • Ensuring developers can access the most comprehensive AI and data services, from Microsoft and our partners

  • Enhancing security, performance and scale needed for AI, even as the requirements of AI apps push past traditional levels

  • And perhaps most importantly, driving clear business results from AI investments


Just as important as what we’re delivering is how we’re making all of this possible: through deeper integration across our flagship AI, Data, App and developer services, by embracing top ISV services across the AI toolchain, embedding AI into core processes to strengthen and streamline development and operations, and taking the knowledge we’ve gained running some of the most advanced intelligent applications in production and converting this to best practices built into the products themselves.


 


For a deeper look at how we’re creating the best environment to build intelligent applications, please read “Maximizing Joy and Minimizing Toil with Great Developer Experiences” by Amanda Silver, Corporate VP and Head of Product for Microsoft’s Developer Division. And for a complete view of our new innovation across AI, Data, Apps and Developers, read through “From Code to Production: New Ways Azure Helps You Build Transformational AI Experiences” from Jessica Hawk, Corporate Vice President of Data, AI, Digital Applications Marketing.


 


Microsoft Build 2024


Let’s take a closer look at some of the innovations coming to Azure’s AI application platform.


 


GitHub Copilot for Azure


GitHub is the world’s most popular developer platform, trusted by over 100 million developers. Combining this developer platform with Azure gives every developer an end-to-end experience optimized for AI applications. This week we’re announcing GitHub Copilot for Azure, enabling developers to build, troubleshoot and deploy applications on Azure all from within Copilot Chat in Visual Studio or Visual Studio Code. This helps every developer, even those new to Azure, build fast, accessing services from Azure and our partners through the developer tools you already love and deploy to a platform optimized for the specific needs of your application. With GitHub Copilot for Azure, you can now use natural language to enable an even tighter developer loop with Azure. Just another example of how we’re removing friction across the entire development and cloud lifecycle, so developers can focus on building and running great apps.


 


For more information on GitHub Copilot for Azure, explore the technical blog


 


AKS Automatic


Within our application platform services, we’re introducing Azure Kubernetes Service Automatic, the easiest managed Kubernetes experience to take AI apps to production. In public preview, AKS Automatic builds on our expertise running some of the largest and most advanced Kubernetes applications in the world, from Microsoft Teams to Bing, Xbox online services, M365 and GitHub Copilot to create best practices that automate everything from cluster set up and management to performance and security safeguards and policies. As a developer you now have access to a self-service app platform that helps you go from code to Kubernetes in minutes without giving up the ability to use the Kubernetes API and staying aligned with the cloud native ecosystem. With AKS Automatic you can focus on building great code, knowing that your app will be running securely with the scale, performance and reliability it needs to support your business.


 


For more information on AKS Automatic, explore the technical blog and attend this session to learn more.


 


Delivering the Complete AI Toolchain


Microsoft is bringing developers the most complete AI toolchain, with the flexibility to work with the best, most innovative services available, integrated into a unified developer experience. At Build we’re launching new and expanded relationships with providers of leading services that are key to generative AI applications. Even more, through unique integrations and expanded pre-built application templates we’re creating a unified, native experience across key partner services within our developer tools including GitHub Copilot and Visual Studio Code as well as into our application platform services such as Azure Container Apps.


 


Arize offers an AI observability and LLM evaluation platform that helps AI developers and data scientists monitor, troubleshoot, and evaluate LLM models. This offering is critical to observe and evaluate applications for performance improvements in the build-learn-improve development loop. Arize AI’s observability platform features native support for Azure customers leveraging Azure AI Studio, offering complete visibility into every layer of an LLM-based software system: the application, the prompt, and the response.


 


Read more about Arize’s integration with Azure AI Studio here.


 


Hugging Face, the most popular AI community with millions of users worldwide, offers Spaces, a simple and powerful tool that enables developers to showcase their models and datasets. Microsoft and Hugging Face are launching an expanded partnership that enables developers to open and use their Hugging Face Spaces environment directly within Visual Studio Code, delivering an integrated developer experience.


 


Read more about Hugging Face Spaces integration with Visual Studio Code here. Create a Space in dev mode here.


 


LangChain is an open-source framework that helps developers build LLM (Large Language Model) applications from prototype to production. With LangChain, you can build context-aware, reasoning agents that integrate with external data sources and tooling to make informed, real-world decisions. LangChain now integrates with Azure Container Apps Dynamic Sessions to enable secure code execution. This supports our announcement of Azure Container Apps Dynamic Sessions that includes LLM-generated code execution with easy access to short-lived, secure sandboxes.


 


Read more about the LangChain integration here.


 


LlamaIndex is a leading data framework for building LLM applications, connecting large language models (LLMs) to custom data sources . Their framework provides a complete set of tools for preparing and querying data for some of the most popular AI app patterns, including RAG. LlamaIndex now integrates with Azure Container Apps Dynamic Sessions to enable secure code execution. This supports our announcement of Azure Container Apps Dynamic Sessions that includes LLM-generated code execution with easy access to short-lived, secure sandboxes.


 


Read more about the LlamaIndex integration here.


 


Pinecone has built a strong following with developers for its serverless vector database that helps create and scale AI applications faster and cheaper. Pinecone’s vector database enables improved, highly accurate results from generative AI applications while its low latency delivers the performance needed for real time generative AI applications.


To help developers easily manage their vector database from within their codebase, Pinecone is delivering a GitHub Copilot agent plugin enabling developers to interact with the plugin for help with vector databases within their codebase, a massive efficiency leap for developers building generative AI applications.


 


Read more about Pinecone’s GitHub Copilot agent here.


 


Building your startup with Azure


Microsoft for Startups Founders Hub supports startups with free access to the latest generative AI models, up to $150,000 Azure credits for Azure AI, as well as unlimited 1:1 meetings with Microsoft experts to help solve business and technical issues. Now, Microsoft for Startups Founders Hub is making it even easier and faster for startups to build AI apps.  


 


Build with AI


Quickly create and launch AI solutions tailored to your startup’s needs with a curated set of flexible templates, saving you time and resources. Templates designed for common AI use cases open in your development tool of choice and provide everything you need to understand, edit, and deploy them, as well as tips for customizing and troubleshooting.


 


AI assistant in Founders Hub*


Easily accessible via a button on the navigation bar or through shortcuts on your homepage, this assistant provides recommended questions or prompts users to ask their own questions and get comprehensive support at each stage of your startup journey.


 


Expanding Startup Success: These new capabilities are just a few of the ways we are expanding the Founders Hub platform to help startups accelerate AI innovation. Today, Nixtla, Bria, and Gretel Labs, all part of the Microsoft for Startups program, made significant announcements, and we are looking forward to even more innovation from our Founders Hub startups. 


Read more about our investments in the startup community in this blog from Annie Pearl, CVP Ecosystems at Microsoft.


 


* AI assistant in Founders Hub is currently in private preview and available in select regions.


 


AI Assisted Operations in AKS


As organizations expand Kubernetes deployments, many have added more Kubernetes clusters to their app platforms, introducing the need to manage hundreds to thousands of clusters efficiently. However, there is generally no programmatic way to place workloads into the clusters intelligently leaving Ops teams to look for a more sophisticated orchestration engine.  


 


To help facilitate scheduling workloads intelligently across different clusters to maximize resource usage based on heuristics such as cost and availability of resources, intelligent workload scheduling in Azure Kubernetes Fleet Manager is now available in public preview. IT Operations teams can optimize resource management and workload placement for their applications by letting AKS maximize their performance and scale. Security-conscious Ops teams can use deployment safeguards to make it easier to apply and enforce the right policies on their apps, thereby strengthening governance and cost management at scale.


 


We are also announcing Kubernetes AI Toolchain Operator (KAITO) in Public Preview as an AKS add-on. The KAITO add-on enables running specialized machine learning workloads like large language models (LLMs) on AKS more cost-effectively and with less manual configuration. AKS is also adding support for Windows Graphical Processing Units (GPU). This eliminates the need for manual installation and enables workloads such as training deep learning models, visualizing complex data, and more. Additionally, with the help of AI in workload placement and scheduling, customers can now operate Windows workloads on AKS with greater flexibility.


 


For more information on AI Assisted Operations in AKS, explore the technical blog.


 


Azure Container Apps dynamic sessions


Dynamic sessions in Azure Container Apps, now in public preview, enables AI app developers to instantly run LLM-generated code or extend/customize SaaS applications in an on-demand secure sandbox. Developers running in a multitenant environment can mitigate security risks by executing this untrusted code in an isolated environment, while leveraging the event-driven serverless scale provided by Azure Container Apps. Dynamic sessions are reusable and are automatically cleaned up greatly reducing management overhead and costs.


 


For more information on ACA Dynamic Sessions, explore the technical blog and  attend the session “BRK131: Serverless architectures: Effortless Intelligent Apps at extreme scale”  


 


API Management for AI Apps


The growth of Generative AI based applications has increased the need to secure and manage access to these Gen AI APIs. Azure API Management is a market leading API Management solution protecting over a million APIs and processing over a trillion API requests every month. To address the security and management challenges for Azure OpenAI Services endpoints, we are excited to announce the introduction of GenAI Gateway capabilities in Azure API Management.


 


Customers can now import all Azure OpenAI endpoints into the Azure API Management platform with a single click and can benefit from API Management’s built-in authentication policies to secure access to these endpoints. As the usage of Azure OpenAI APIs increases, customers can leverage built-in load balancing, rate limiting based on tokens, and benefit from out-of-the-box observability support in Azure API Management. Customers can also use Semantic caching to connect with a compatible caching solution such as Azure Redis Enterprise to reduce token consumption and improve response performance.


 


For more information on API Management for AI Apps, explore the technical blog and this Build session.


 


Azure Functions for OpenAI


Azure Functions is also launching new features to provide more flexibility and extensibility for AI applications.


 


Azure Functions extension for OpenAI, now in public preview, enables developers to build Functions that integrate with OpenAI, as well as Azure OpenAI. Developers can use this extension to build AI applications using .NET, Java, Python, and many other languages for Retrieval Augmented Generation, Text completion, Chat assistants and so on.


 


Additionally, we are making our Flex Consumption Plan generally available. This plan is for applications requiring event driven scale with negligible cold-start latency and always-ready instances making it a great option for AI apps.


 


For more information on the enhancements in Azure Functions, explore , explore the technical blog and  attend the session “BRK131: Serverless architectures: Effortless Intelligent Apps at extreme scale”


 


Learn more at Microsoft Build 2024 



 


[1] https://www.idc.com/getdoc.jsp?containerId=US51953724


[2] Source: IDC white paper, sponsored by Microsoft, Exploring the Benefits of Cloud Migration and Modernization.


[3] https://github.blog/2023-06-27-the-economic-impact-of-the-ai-powered-developer-lifecycle-and-lessons-from-github-copilot/


Published by satonaoki

Leave a comment