A continuation of the previous blog post.

Can we shrink the size of the model using fine-tuning, LoRa and quantization to fit on a Raspberry Pi 4 with reasonable performance?​

We monetize on running the server for private re-training? Private Models? (does it matter)? Or in exchange for DP, we get anonymized data sets back for training?

A continuation of the previous blog post.

Can we shrink the size of the model using fine-tuning, LoRa and quantization to fit on a Raspberry Pi 4 with reasonable performance?​

Prerequisite thoughts​

• include "not_going_to_agi_soon.md"
• include "Private agents are good!"

What is the problem I am solving?​

When describing my cloud-free internet set up at home, a distinguished former manager congratulated me on my pro-bono secondary IT job and wondered how I'd find the time. As I was still in the honeymoon phase and having fun, I generally dismissed the comment.

A few years on, the risk of "set-it-and-forget-it" is real. Recent advances in machine learning make it easier to lower the burden of keeping code up to date at home, but they rely on public cloud LLMs to operate which has issues with:

1. Data privacy
2. Data sovereignty
3. Data security

Lots of new information to assimilate, this first-of-many post focuses on private agency related thoughts including putting the puzzle together for Private Agency for My House.

Soumith Chintala

Ingredients to the bake Private Agency in the home?​

Ignorance is dangerous, so let's take a look at what are the known-knowns, known-unknowns, and unknown-unknowns.

Going to California for NVIDIA GTC 2025, what are the areas of focus for privacy?

To be updated​

The AI Engineering Summit was a definite eye-opener to the speed with which IT is transforming mundane "busy" work, and the startup cost for exploring new ideas.

Take aways for a private agent:​

• Do Agents need to be local to be private?
• Locally on a macbookpro
• Hosted in a secure enclave on AWS / Azure / GCP
• Or can some sort of formal proof be done to leave a multi-tenant agent in the cloud with data privacy?
• For either of the above, how to manage the balance of cost ?
• How are we protecting state?
• Where are we storing state?

======= Caveat Emptor: This is a public blog of what probably should be a private note.

ff0c541 (Additions to new life`)

Machine Specification:​

• CPU: Mac M2 Pro
• RAM: 96GB DDR4
• Storage: 1TB NVMe SSD

Tonight turned into an unexpected journey through the rabbit holes of self-hosted technologies. I decided to venture out of my comfort zone, and boy, was it an enlightening experience! Here's a rundown of gotchas:

<<<<<<< HEAD

Respect to the makers and tinkers working to make Technology free and private.

The focus of this stack are for the class of IT worker with less time to understand the beauty of the low-level implementation than the smart-home-function they would like to accomplish. Over the course of 2022, the goal is to publish blueprints of “How I did that?” with the goal to incorporate a “Hey Jimmy” home assistant.

For the Electricians and hobbyists that are looking to differentiate themselves in the market, we offer training and support on how to choose, configure, install and support smart home technology.