<titledata-rh="true">Running an AI application | Podman Desktop</title><metadata-rh="true"name="viewport"content="width=device-width,initial-scale=1"><metadata-rh="true"name="twitter:card"content="summary_large_image"><metadata-rh="true"property="og:image"content="https://podman-desktop.io/img/banner_podman-desktop.png"><metadata-rh="true"name="twitter:image"content="https://podman-desktop.io/img/banner_podman-desktop.png"><metadata-rh="true"property="og:url"content="https://podman-desktop.io/tutorial/running-an-ai-application"><metadata-rh="true"property="og:locale"content="en"><metadata-rh="true"name="docusaurus_locale"content="en"><metadata-rh="true"name="docsearch:language"content="en"><metadata-rh="true"name="docusaurus_version"content="current"><metadata-rh="true"name="docusaurus_tag"content="docs-tutorial-current"><metadata-rh="true"name="docsearch:version"content="current"><metadata-rh="true"name="docsearch:docusaurus_tag"content="docs-tutorial-current"><metadata-rh="true"property="og:title"content="Running an AI application | Podman Desktop"><metadata-rh="true"name="description"content="Running an AI application using the UI."><metadata-rh="true"property="og:description"content="Running an AI application using the UI."><metadata-rh="true"name="keywords"content="podman desktop,podman,AI application"><linkdata-rh="true"rel="icon"href="/img/favicon.ico"><linkdata-rh="true"rel="canonical"href="https://podman-desktop.io/tutorial/running-an-ai-application"><linkdata-rh="true"rel="alternate"href="https://podman-desktop.io/tutorial/running-an-ai-application"hreflang="en"><linkdata-rh="true"rel="alternate"href="https://podman-desktop.io/tutorial/running-an-ai-application"hreflang="x-default"><linkdata-rh="true"rel="preconnect"href="https://MR01ANKQ9S-dsn.algolia.net"crossorigin="anonymous"><linkrel="alternate"type="application/rss+xml"href="/blog/rss.xml"title="Podman Desktop RSS Feed">
<linkrel="alternate"type="application/atom+xml"href="/blog/atom.xml"title="Podman Desktop Atom Feed">
<p>This tutorial covers the following end-to-end tasks required to run and verify an Artificial Intelligence (AI) application on your local machine:</p>
<ul>
<li>Installing the extension</li>
<li>Downloading a model</li>
<li>Creating a service</li>
<li>Optional: Creating a playground</li>
<li>Running a recipe</li>
<li>Verifying the AI application</li>
</ul>
<p>Before going through this tutorial, familiarize yourself with the usage of the following sections on the AI Lab page:</p>
<li><strong>Catalog</strong>: Helps in downloading or importing a model.</li>
<li><strong>Services</strong>: Helps in creating an inference server that runs in a container to serve a large language model (LLM) using an OpenAI-compatible API. After creating a service, you can access its details and generate its client code in any programming languages to integrate your application.</li>
<li><strong>Playgrounds</strong>: Helps in experimenting with model parameters and accuracy to identify the best model and parameters for your use case. Each playground denotes a customized environment to experiment with a model.</li>
<li><strong>Recipes Catalog</strong>: Helps in accessing a catalog of recipes, such as ChatBot, audio-to-text, code generation, and summarizer. Each recipe denotes an AI application catering to a specific use case. You can use a recipe to start an AI application. You can also clone a recipe to your local machine for editing purposes.</li>
<li><strong>Running</strong>: Helps in accessing all running AI applications.</li>
</ol>
<p>This tutorial focuses on running a ChatBot AI application.</p>
<h2class="anchor anchorWithStickyNavbar_JmGV"id="before-you-begin">Before you begin<ahref="#before-you-begin"class="hash-link"aria-label="Direct link to Before you begin"title="Direct link to Before you begin"></a></h2>
<li><ahref="/docs/podman/creating-a-podman-machine">A running Podman machine</a> with at least 6 GB memory. If the Podman machine does not have enough memory, you get a notification to update its memory configuration while downloading a model or running a recipe:
<imgdecoding="async"loading="lazy"alt="notification to update podman machine configuration"src="/assets/images/update-machine-config-169e3fa45ac02ad207b99caaf0124408.png"width="1272"height="409"class="img_SS3x"></li>
<li>A developer role.</li>
</ul>
<h2class="anchor anchorWithStickyNavbar_JmGV"id="installing-the-extension">Installing the extension<ahref="#installing-the-extension"class="hash-link"aria-label="Direct link to Installing the extension"title="Direct link to Installing the extension"></a></h2>
<ol>
<li>Go to <strong>Extensions > Catalog</strong>.</li>
<li>Find the Podman AI Lab extension using the search box.</li>
<li>Click the <strong>Install</strong> icon to install the extension.
<imgdecoding="async"loading="lazy"alt="install the extension"src="/assets/images/downloading-ai-lab-ext-24af9218d5d28b262f52ef5f17e20d42.png"width="1962"height="634"class="img_SS3x"></li>
<li>Select the <strong>Installed</strong> tab to check the extension is active, and check that a new AI Lab component is added in the left navigation pane.
<h2class="anchor anchorWithStickyNavbar_JmGV"id="downloading-a-model">Downloading a model<ahref="#downloading-a-model"class="hash-link"aria-label="Direct link to Downloading a model"title="Direct link to Downloading a model"></a></h2>
<ol>
<li>
<p>Click <strong>AI Lab</strong> in the left navigation pane.</p>
</li>
<li>
<p>In the Catalog section, click the <strong>Download</strong> icon corresponding to the model you want to use.
<h2class="anchor anchorWithStickyNavbar_JmGV"id="creating-a-service">Creating a service<ahref="#creating-a-service"class="hash-link"aria-label="Direct link to Creating a service"title="Direct link to Creating a service"></a></h2>
<ol>
<li>
<p>Click <strong>AI Lab</strong> in the left navigation pane.</p>
</li>
<li>
<p>In the Services section, click <strong>New Model Service</strong>.
<imgdecoding="async"loading="lazy"alt="create a new model service"src="/assets/images/create-new-model-service-ea8c2365c8c0bd53be04353d4bbbadbe.png"width="1086"height="750"class="img_SS3x"></p>
</li>
<li>
<p>Select the model from the dropdown list.</p>
</li>
<li>
<p>Optional: Edit the port number of the container that is created for the service.</p>
</li>
<li>
<p>Click <strong>Create service</strong>.
<imgdecoding="async"loading="lazy"alt="create a new model service"src="/assets/images/create-a-service-fe22540049e76cdd058004bc3abaa195.png"width="1536"height="630"class="img_SS3x"></p>
</li>
<li>
<p>Click <strong>Open service details</strong>.
<imgdecoding="async"loading="lazy"alt="create a new model service"src="/assets/images/open-service-details-74e703c418efbc20f7ec65e9a14acb8e.png"width="1536"height="1024"class="img_SS3x"></p>
</li>
<li>
<p>View the service details, such as model name, server address, and client code.
<divclass="theme-admonition theme-admonition-note admonition_WCGJ alert alert--secondary"><divclass="admonitionHeading_GCBg"><spanclass="admonitionIcon_L39b"><svgviewBox="0 0 14 16"><pathfill-rule="evenodd"d="M6.3 5.69a.942.942 0 0 1-.28-.7c0-.28.09-.52.28-.7.19-.18.42-.28.7-.28.28 0 .52.09.7.28.18.19.28.42.28.7 0 .28-.09.52-.28.7a1 1 0 0 1-.7.3c-.28 0-.52-.11-.7-.3zM8 7.99c-.02-.25-.11-.48-.31-.69-.2-.19-.42-.3-.69-.31H6c-.27.02-.48.13-.69.31-.2.2-.3.44-.31.69h1v3c.02.27.11.5.31.69.2.2.42.31.69.31h1c.27 0 .48-.11.69-.31.2-.19.3-.42.31-.69H8V7.98v.01zM7 2.3c-3.14 0-5.7 2.54-5.7 5.68 0 3.14 2.56 5.7 5.7 5.7s5.7-2.55 5.7-5.7c0-3.15-2.56-5.69-5.7-5.69v.01zM7 .98c3.86 0 7 3.14 7 7s-3.14 7-7 7-7-3.12-7-7 3.14-7 7-7z"></path></svg></span>note</div><divclass="admonitionContent_pbrs"><p>By default, you view the client code in the cURL programming language. You can customize the client code based on your needs.</p></div></div>
</li>
<li>
<p>Optional: In the Client code section, select the programming language or framework from the dropdown list to view the updated code.</p>
</li>
<li>
<p>Click the <strong>Close</strong> icon on the right side of the page.</p>
</li>
<li>
<p>View the newly created service on the same page.
<imgdecoding="async"loading="lazy"alt="new service on the page"src="/assets/images/newly-created-service-226100da3e06b054b4355ff0c1ee9be1.png"width="1536"height="266"class="img_SS3x"></p>
</li>
<li>
<p>Optional: View the service container by opening the <strong>Containers</strong> page from the left navigation pane.
<imgdecoding="async"loading="lazy"alt="new service container"src="/assets/images/running-service-in-a-container-59d98e3e8c667f91e4e54c97fea80616.png"width="1982"height="506"class="img_SS3x"></p>
</li>
</ol>
<h2class="anchor anchorWithStickyNavbar_JmGV"id="optional-creating-a-playground">Optional: Creating a playground<ahref="#optional-creating-a-playground"class="hash-link"aria-label="Direct link to Optional: Creating a playground"title="Direct link to Optional: Creating a playground"></a></h2>
<p>Perform this task only if you want to experiment with the model before using it to start your AI application.</p>
<ol>
<li>
<p>Click <strong>AI Lab</strong> in the left navigation pane.</p>
</li>
<li>
<p>In the Playgrounds section, click <strong>New Playground</strong>.
<imgdecoding="async"loading="lazy"alt="create a new playground"src="/assets/images/new-playground-f7f1112477f363057fb96e4f6a9b81b8.png"width="1086"height="728"class="img_SS3x"></p>
</li>
<li>
<p>Optional: Enter the name of the playground.</p>
<p>View the newly created playground on the same page.
<imgdecoding="async"loading="lazy"alt="newly created playground"src="/assets/images/newly-created-playground-4f6f926a0c0ab6e09417233df7902d25.png"width="1268"height="276"class="img_SS3x"></p>
</li>
<li>
<p>Click the name of the playground to edit the model settings for experiment:</p>
<ol>
<li>
<p>Edit the value of the model parameters.
<imgdecoding="async"loading="lazy"alt="editing model parameters"src="/assets/images/editing-model-parameters-4e74800edfc96ddc871e30d5043a820c.png"width="1572"height="792"class="img_SS3x"></p>
</li>
<li>
<p>Click the <strong>Edit system prompt</strong> icon to add a system prompt and save it.
<imgdecoding="async"loading="lazy"alt="edit system prompt icon"src="/assets/images/edit-system-prompt-icon-4fd921890f57cb4f68f4d69e9e3d92d0.png"width="1304"height="379"class="img_SS3x"></p>
</li>
<li>
<p>Enter your question in the prompt box and analyze the accuracy of the answer.
<imgdecoding="async"loading="lazy"alt="defining a system prompt"src="/assets/images/defining-a-system-prompt-8671ce78a8b33a2a3a8eb03a750a2c7a.png"width="1572"height="1292"class="img_SS3x"></p>
</li>
</ol>
</li>
</ol>
<h2class="anchor anchorWithStickyNavbar_JmGV"id="running-a-recipe-chatbot-ai-application">Running a recipe: ChatBot AI application<ahref="#running-a-recipe-chatbot-ai-application"class="hash-link"aria-label="Direct link to Running a recipe: ChatBot AI application"title="Direct link to Running a recipe: ChatBot AI application"></a></h2>
<ol>
<li>
<p>Click <strong>AI Lab</strong> in the left navigation pane.</p>
</li>
<li>
<p>In the Recipes Catalog section, click <strong>More details</strong> in the ChatBot recipe tile. The recipe summary page opens.
<imgdecoding="async"loading="lazy"alt="more details of a recipe"src="/assets/images/more-details-of-recipe-f570ab4ce1000ad4cc34f792f169c98a.png"width="1272"height="714"class="img_SS3x"></p>
</li>
<li>
<p>Click <strong>Start</strong>. The Start recipe page opens.
<imgdecoding="async"loading="lazy"alt="starting a recipe"src="/assets/images/starting-a-recipe-dff7eadac53d4f9b266bba8a1f3aef78.png"width="1268"height="674"class="img_SS3x"></p>
</li>
<li>
<p>Select a model from the dropdown list.</p>
<divclass="theme-admonition theme-admonition-note admonition_WCGJ alert alert--secondary"><divclass="admonitionHeading_GCBg"><spanclass="admonitionIcon_L39b"><svgviewBox="0 0 14 16"><pathfill-rule="evenodd"d="M6.3 5.69a.942.942 0 0 1-.28-.7c0-.28.09-.52.28-.7.19-.18.42-.28.7-.28.28 0 .52.09.7.28.18.19.28.42.28.7 0 .28-.09.52-.28.7a1 1 0 0 1-.7.3c-.28 0-.52-.11-.7-.3zM8 7.99c-.02-.25-.11-.48-.31-.69-.2-.19-.42-.3-.69-.31H6c-.27.02-.48.13-.69.31-.2.2-.3.44-.31.69h1v3c.02.27.11.5.31.69.2.2.42.31.69.31h1c.27 0 .48-.11.69-.31.2-.19.3-.42.31-.69H8V7.98v.01zM7 2.3c-3.14 0-5.7 2.54-5.7 5.68 0 3.14 2.56 5.7 5.7 5.7s5.7-2.55 5.7-5.7c0-3.15-2.56-5.69-5.7-5.69v.01zM7 .98c3.86 0 7 3.14 7 7s-3.14 7-7 7-7-3.12-7-7 3.14-7 7-7z"></path></svg></span>note</div><divclass="admonitionContent_pbrs"><p>The models listed with a star are recommended to use with the recipe.</p></div></div>
</li>
<li>
<p>Click <strong>Start ChatBot recipe</strong>. This step might take some time to pull the recipe, copy the model to your Podman machine, start the inference server, load configurations, and create the application.
<imgdecoding="async"loading="lazy"alt="starting a chatbot recipe"src="/assets/images/starting-a-chatbot-recipe-2563c94d79b9668908ce96141fbbe54d.png"width="1268"height="652"class="img_SS3x"></p>
</li>
<li>
<p>Click <strong>Open details</strong> after the processing completes.
<p>Optional: View the running AI application pod by opening the <strong>Pods</strong> page from the left navigation pane.
<imgdecoding="async"loading="lazy"alt="pod running AI app"src="/assets/images/pod-running-ai-app-a9d36d4bd79935ea6ba153d5d80efebb.png"width="1732"height="424"class="img_SS3x"></p>
</li>
</ol>
<h2class="anchor anchorWithStickyNavbar_JmGV"id="verifying-the-ai-application">Verifying the AI application<ahref="#verifying-the-ai-application"class="hash-link"aria-label="Direct link to Verifying the AI application"title="Direct link to Verifying the AI application"></a></h2>
<ol>
<li>
<p>Go to <strong>AI Lab > Running</strong> using the left navigation pane.</p>
</li>
<li>
<p>Click the <strong>Open AI App</strong> icon to open the ChatBot application locally in a browser.
<imgdecoding="async"loading="lazy"alt="opening the AI application"src="/assets/images/open-ai-application-bc696ad523aa59a78734fee241d23b64.png"width="1542"height="306"class="img_SS3x"></p>
</li>
<li>
<p>Enter your question in the chat box and analyze the accuracy of the answer.
<imgdecoding="async"loading="lazy"alt="interacted with chatbot application"src="/assets/images/interacted-with-chatbot-application-e44958a60f1560c05f4b993289e36c58.png"width="1368"height="936"class="img_SS3x"></p>
