ZNDET’s key takeaways
- If you can’t find the app you need, build it.
- You can do this for free using open-source tools.
- This is done using Ollama and VS Code.
Over the weekend I decided I needed a specific writing tool…that didn’t exist. Everything I tried was either too cumbersome, lacked the features I needed, or was too expensive.
Also: I’m leaving ChatGPT for a free, private and local AI called Ollama – here’s why
It was then I heard about a fellow author (Hugh Howey) who started his own writing app (NEO) about ten years ago. He was a software developer at the time, and he felt the same way I did: most word processors were too bloated for an author’s needs.
For me, when I write a first draft, I only need certain functions (magic / grammar check, dashboard, formatting, export / import, light / dark modes, etc.) and nothing else. Scrivener was too expensive, and the traditional word processors were too many, or tied to larger subscription models.
I tried Howey’s NEO and liked it, but I wanted to craft something specific for myself. The problem is, although I am “developer-adjacent”, I am not a full-fledged developer. I can work with Python, CSS, HTML, and a bit of JavaScript, and I knew that wouldn’t cut it for my use case.
Also: Worried about AI’s high energy needs? Avoiding chatbots won’t help – but 3 things might
So I switched to locally installed AI. I knew exactly how to describe the app I wanted, so I thought I’d give it a try and see if it succeeded which I couldn’t. The catch was that I didn’t want to pay for it. I’m a writer, after all, and we’re not exactly known for rolling in dough.
Good thing I know how to work with locally installed AI, that’s how I pulled this off. Let me show you how I did it.
The requirements
You can’t just slap the necessary AI apps on any old system and expect it to be able to churn out a fully developed app. You need some pretty beefy hardware. Here is what I would say is the bare minimum for the project:
- RAM: 32 GB (I have 64 on the machine I use).
- GPU: Nvidia is best.
- CPU: At least a 64-bit CPU with AVX2 instruction set support
- Local AI: Ollama
- Models: Chat – Llama 3.1 8B, Autocomplete – Qwen2.5-Coder, Edit – Llama 3.1 8B, Application – Llama 3.1 8B, Embed – Transformer.js, Agent – gemma4: latest.
- IDE: VS Code
Install Ollama
I install Ollama on a server in my LAN. If you don’t have a spare machine for this, you can install it on the same machine running VS Code. Just make sure that any machine you use meets the requirements.
Also: Want local vibe coding? This AI stack could replace Claude Code and Codex – for free
If you are on Linux or MacOS, installing Ollama is a single command, which is:
curl -fsSL | shIf you are on Windows, this command would be:
irm | iexDownload the necessary models
Next, you need to draw the necessary models. Here are the commands for each:
ollama pull llama3.1:8Bollama pull qwen2.5-coder:1.5bollama pull gemma4Configure Ollama
You now need to configure Ollama to accept connections from your LAN. Out of the box it only accepts requests from your local machine. I will show how to do this on Linux.
First, you need to open the correct configuration file with the command:
sudo nano /etc/systemd/system/ollama.serviceUnder the [service] section, add the following:
Environment="OLLAMA_HOST=0.0.0.0"Save and close the file.
Reload the systemctl daemon with the command:
sudo systemctl daemon-reloadRestart Ollama with the command:
sudo systemctl restart ollamaOllama now accepts requests from your LAN.
Install and configure VS Code
It’s time to install VS Code, which is the IDE I used to build my app (quillmarker). Installing VS Code on Linux is done via the command line. First, download the .deb file (for Ubuntu-based distributions) or the .rpm file (Fedora-based distributions). Once you have downloaded the file, install it using one of the following commands:
Ubuntu-based: sudo dpkg -i code*.debFedora-based: sudo rpm -i code*.rpmIf you use MacOS or Windows, download the installation file for your respective OS, double-click the file and go through the installation wizard.
Also: How to feed my files to a local AI for better, more relevant answers
With VS Code installed, it must be configured to use the locally installed AI. Before you can do that, you need to install the Continue extension in VS Code. To do so, click on the Extensions icon in the left sidebar. In the search bar, type Continue. When the entry appears, click Install.
If Continue has been added, it must be configured. To open the configuration editor, you will find a new icon in the sidebar; click that and then click Models.
This is where it gets slightly complicated. In the list of models, you can choose what you need from each drop-down. Then click on the gear icon for the chat model, which will open the config.yaml file. In this file you will see the configuration for chat, edit, apply, as well as autocomplete and embed. What you need to do is change the line:
apiBase: "to:
apiBase:"Where IP_OF_OLLAMA is the IP address of the machine hosting Ollama.
Also: How to run DeepSeek AI locally to protect your privacy – 2 easy ways
You then need to add this line to the other two entries (autocomplete and embed) in the same position within each block, so it looks like this:
name: Main Config
version: 1.0.0
schema: v1
models:
- name: Llama 3.1 8B
provider: ollama
model: llama3.1:8b
apiBase: "
roles:
- chat
- edit
- apply
- name: Qwen2.5-Coder 1.5B
provider: ollama
model: qwen2.5-coder:1.5b-base
apiBase: "
roles:
- autocomplete
- name: Nomic Embed
provider: ollama
model: nomic-embed-text:latest
apiBase: "
roles:
- embed
Make sure to change the IP address above to that of the machine on your LAN hosting Ollama.
Click the X to the right of the Config.yaml tab and, when prompted, click Save.
You’re almost done.
Now it’s time to customize the agent. Click on the dropdown to the right of Agent in the lower right corner of VS Code. Click on the gear icon in the dropdown, and you should see the list of available models (which should include gemma4). In the far-right column associated with Ollama, click on the gear icon and select Open in Language Models (JSON).
Jack Wallen/ZDNETThis will open a new tab in the VS Code Viewing window for the chat.languageModels.json file.
Make sure the URL line reads:
"url": "Where IP_OF_OLLAMA is the IP address for your Ollama server.
Click the X on the right edge of this tab and click Save when prompted.
Select gemma4 from the agent dropdown, and you are now ready to build your app.
Build your app
If you’ve used AI before, this should be familiar. In the Describe what to build window, enter the description of the app you want VS Code to build; Make sure you are as specific as possible, including all the features you want. For example, when I needed quill markers to include formatting, I added “formatting options similar to those of LibreOffice.” If you are not specific, or you leave out features, you will need to iterate the app again to add them.
Also: I tried Sanctum’s local AI app, and it’s exactly what I needed to keep my data private
This will take some time, and you will need to give VS Code permission to do certain things (like run commands) as it builds. You can go away and do other things, but when you come back, it may ask you to approve a permission. Don’t worry, VS Code is waiting for you.
The first build took about two hours. After that I would have to iterate it again because I might have forgotten a feature or wanted something sophisticated. All in all, I believe it took about seven iterations to complete. Once it was done, I found the Linux AppImage (that’s what I asked to create it, so it can be used on any Linux distribution) in ~/quillmarker/releases. After installation I started writing.
It’s a long process, but in the end it was worth it. I’m using quill markers for my next novel, and I’ve found it a joy to write with.
See full bio