{"id":113,"date":"2024-04-26T19:32:17","date_gmt":"2024-04-27T02:32:17","guid":{"rendered":"http:\/\/Macdaddy4sure.com\/?p=113"},"modified":"2024-04-26T19:37:48","modified_gmt":"2024-04-27T02:37:48","slug":"building-a-powerful-ai-ready-server-combining-llm-whisper-and-linux","status":"publish","type":"post","link":"http:\/\/macdaddy4sure.ai\/index.php\/2024\/04\/26\/building-a-powerful-ai-ready-server-combining-llm-whisper-and-linux\/","title":{"rendered":"Building a Powerful AI-Ready Server: Combining LLM, Whisper, and Linux"},"content":{"rendered":"\n<p><strong>Introduction<\/strong><\/p>\n\n\n\n<p>As AI technology continues to advance at a rapid pace, building a powerful server capable of handling demanding AI<br>workloads is becoming increasingly important. In this post, I&#8217;ll walk you through my journey of creating an<br>artificial intelligence (LLM) and Whisper dedicated server using an old Windows 2016 license and Ubuntu. This<br>setup will provide a robust platform for running AI models, processing large datasets, and integrating with other<br>tools.<\/p>\n\n\n\n<p><strong>Gathering the Components<\/strong><\/p>\n\n\n\n<p>To start, I had an old Windows Server 2016 license lying around, which provided the foundation for our server.<br>Additionally, I obtained a copy of Ubuntu 24.04 Server. I downloaded the LLM (Large Language Model) and Whisper AI models from their respective repositories.<\/p>\n\n\n\n<p><strong>Setting Up the Server<\/strong><\/p>\n\n\n\n<p>First, I installed Ubuntu on my old Windows Server hardware using a USB installer drive. The installation process<br>was straightforward, and I chose to install the 64-bit version of Ubuntu. Once installed, I configured the network<br>settings and set up the root password for the server.<\/p>\n\n\n\n<p>Next, I enabled Secure Sockets Host (SSH) in Ubuntu to access the server remotely. This allowed me to control<br>the machine from anywhere, making it easier to manage and monitor the server.<\/p>\n\n\n\n<p><strong>Configuring the Server<\/strong><\/p>\n\n\n\n<p>To optimize the server for AI workloads, I made several configuration changes:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>CPU Affinity<\/strong>: To ensure efficient processing of AI models, I set CPU affinity for each model using the<br>taskset command. This allowed me to allocate specific CPUs to each process. I have 24 total threads in my 1U Dual Xeon with 12 threads reserved for the Virtual Machine.<\/li>\n\n\n\n<li><strong>Memory Allocation<\/strong>: I adjusted memory allocation settings to accommodate the increased demand from running<br>multiple AI models simultaneously. 32 GB of DDR3 ECC RAM allocated to the VM.<\/li>\n\n\n\n<li><strong>Network Settings<\/strong>: I optimized network settings to minimize latency and ensure efficient data transfer, and port forwarded nesseasry ports and application profiles in UFW.<\/li>\n<\/ul>\n\n\n\n<p><strong>Testing the Server<\/strong><\/p>\n\n\n\n<p>To test the server, I ran a few AI model simulations using LLM and Whisper. The results were impressive, with the<br>server handling demanding tasks with ease. The Ubuntu-based setup provided excellent performance and stability,<br>allowing me to focus on developing and refining my AI models.<\/p>\n\n\n\n<p><strong>Conclusion<\/strong><\/p>\n\n\n\n<p>In this post, we&#8217;ve explored how to create a powerful AI-ready server by combining an old Windows Server 2016<br>license with Ubuntu. By installing LLM and Whisper AI models, we were able to build a robust platform for<br>processing large datasets and integrating with other tools. This setup provides a solid foundation for anyone<br>looking to develop and deploy AI applications.<\/p>\n\n\n\n<p><strong>Future Development<\/strong><\/p>\n\n\n\n<p>In the future, I plan to explore further optimizations, such as:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>GPU Support<\/strong>: Adding a graphics processing unit (GPU) would significantly enhance performance for AI<br>workloads.<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li>As AI continues to evolve, I&#8217;m excited to see where this setup takes me. Join me on this journey by following my<\/li>\n\n\n\n<li>future updates and exploring new possibilities in AI development!<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Introduction As AI technology continues to advance at a rapid pace, building a powerful server capable of handling demanding AIworkloads is becoming increasingly important. In this post, I&#8217;ll walk you through my journey of creating anartificial intelligence (LLM) and Whisper dedicated server using an old Windows 2016 license and Ubuntu. Thissetup will provide a robust [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-113","post","type-post","status-publish","format-standard","hentry"],"_links":{"self":[{"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/posts\/113","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/comments?post=113"}],"version-history":[{"count":5,"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/posts\/113\/revisions"}],"predecessor-version":[{"id":119,"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/posts\/113\/revisions\/119"}],"wp:attachment":[{"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/media?parent=113"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/categories?post=113"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/macdaddy4sure.ai\/index.php\/wp-json\/wp\/v2\/tags?post=113"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}