Ollama is a tool for running large language models (LLMs) locally on personal computers. It’s designed to make it easy for users to download, run, and interact with various open-source language models. Ollama is a powerful and user-friendly platform for running large language models (LLMs) on local machines. This means that anyone can utilize the power of advanced AI language processing without relying on cloud-based services or expensive infrastructure. The project’s name, Ollama, comes from the phrase “Open Language Models for All,” reflecting its mission to democratize access to AI-driven language tools.
What can Ollama do for you ?
Easy access to Artificial Intelligence (AI) models:
Ollama allows you to run various AI language models on your personal computer. This means you can interact with AI assistants similar to ChatGPT, but privately on your own machine.
Personal AI assistant:
You can use Ollama to have conversations, ask questions, or get help with various tasks. It’s like having a knowledgeable helper available at any time.
Writing aid:
Ollama can help with writing tasks, such as drafting emails, creating content, or proofreading.
Learning tool:
You can use it to explain concepts, learn about new topics, or get summaries of complex information.
I show excerpt of a typical legal document coached in legalese (long winded language preferred by Lawyers) here:
WHEREAS, the Party of the First Part (hereinafter referred to as ‘the Grantor’), for and in consideration of the sum and other good and valuable consideration, the receipt and sufficiency of which is hereby acknowledged, does hereby remise, release, alienate, convey, and confirm unto the Party of the Second Part (hereinafter referred to as ‘the Grantee’), its heirs, successors, and assigns, all right, title, interest, claim, and demand whatsoever of the Grantor, either in law or in equity, in, to, or out of that certain parcel of real property situated in the County of [Insert County Name], State of [Insert State Name], more particularly described in Exhibit A attached hereto and incorporated herein by this reference, together with all the tenements, hereditaments, and appurtenances thereto belonging or in anywise appertaining, to have and to hold the same unto the Grantee, its heirs, successors, and assigns forever, subject to the covenants, conditions, restrictions, and easements of record.
And the same paraphrased into plain English with the prompt – ‘Give a summary in plain English of the above’:
The person who is selling (the “Grantor”) gives up all their rights and ownership of a specific piece of property (described in Exhibit A) to the buyer (the “Grantee”). The Grantor does this because they received a certain amount of money or other valuable things, which they acknowledge as enough payment. The Grantee gets to own the property forever, along with any buildings, fences, or other things that come with it. However, the sale is subject to any agreements or rules that are already recorded (like easements or restrictions) and apply to the property.
Now you know how useful is ollama. It can read your documents, papers and articles and summarize them into a format you can understand easily. But the whole thing is done inside your computer and this way – you don’t share your personal files with anyone outside.
Creative partner:
It can assist with brainstorming ideas for projects, stories, or problem-solving.
Simple setup:
While it does require some basic installation, the process is designed to be straightforward.
Privacy:
Unlike cloud-based AI services, Ollama runs locally, which means your data stays on your computer.
What is Ollama ?
Ollama is an open-source platform that empowers researchers and developers to deploy large language models (LLMs) on their own machines, sans the need for cloud-based infrastructure. Founded by Michael Chiang and Jeffrey Morgan in Palo Alto, California, Ollama is an independent startup that simplifies the process of running models like Meta AI’s Llama, it is crucial to note that Ollama itself is not a product of Meta or any other major tech company. Instead, Ollama serves as a community-driven project that democratizes access to powerful LLMs developed by various organizations.
Now that we know the basics of Ollama, we will explore its salient features here and at the end of this page, we will set up ollama in our laptop/desk top computers to help us in our daily life – writing mails, reports, research information and use ollama along with its well known models as a local repository of knowledge.
The following paragraph explains in simple terms about its key features. Then we get down to the brass tacks of how to install the Ollama right into your laptop.
Key points about Ollama:
Local deployment: It allows users to run LLMs on their own hardware, which can provide more privacy and control compared to cloud-based solutions.
Ease of use: Ollama aims to simplify the process of setting up and running LLMs, making it accessible to a wider range of users.
Model support: It supports various open-source models, including different versions of Llama, GPT-J, and others.
Command-line interface: Users can interact with models through a command-line interface, which can be integrated into various workflows.
API: Ollama provides an API that allows developers to integrate these local LLMs into their applications.
Cross-platform: It’s available for multiple operating systems, including macOS, Linux, and Windows.
Resource management: Ollama is designed to work efficiently on consumer-grade hardware, though performance can vary depending on the model and the user’s system specifications. What sets these models apart is their relatively modest computational requirements compared to other LLMs, making them an attractive option for researchers and developers seeking to harness the power of language models without breaking the bank.
Open-source: The Ollama project itself is open-source, allowing for community contributions and transparency.
Ollama allows users to run LLMs on their local machines, eliminating the need for cloud-based services or expensive infrastructure. This is achieved through a simple and intuitive interface that enables users to easily integrate LLMs into their own applications. The platform supports various programming languages, including Python, Java, and C++, making it accessible to developers from diverse backgrounds.
Setting up Ollama locally:
I have tried ollama in about a dozen computers with varying degree of success. We have used Linux – Centos 7, Windows 10 , Windows 11 and MacOS. We used computers running on Intel 3, Intel 5, Intel 7 CPUs. If your computer has anything less than 4 Cores, you would be better of with Cloud AI. You need at least 16 GB to run many models.
While Ollama generally offers good performance, users with very high performance requirements or those working with the largest models might still need more powerful hardware or specialized solutions. For most users running common models on typical consumer hardware, Ollama provides a good balance of performance and accessibility.
If you are on Windows: Run the following setup file:
https://ollama.com/download/OllamaSetup.exe
If you are on Linux: Initiate the following:
curl -fsSL https://ollama.com/install.sh | sh
After you are done installing, take a look at the following link for the available models:
https://ollama.com/library
Select the model as per your hardware.
We found Phi3 from Microsoft as the easiest model to run on most computers as it has 2.2 GB size. But it has serious limitation : Hallucinates a lot and has poor English skills (makes many spelling mistakes).
The best is Lllama3.1 from Meta. Its size is 4.7 GB and runs well in I5 and above even without a GPU.
Once it is done, it is easy to invoke it from your command prompt by Clicking on START in Windows and type Windows PowerShell.
Then type:
ollama run llama3
The above will start the Ollama with Llama 3 model. You can enter your prompts ( requests) and it will spew out answers to your questions.
You will be surprised how such a powerful tool escaped your attention all these days.