I iterate on NVIDIA aarch64 hardware every day. Spark DGX is server for my app, hosting agent and prototype for a mesh network node that can be deployed and scaled to VPS at a push of a button. AI-first dev velocity increased, costs decreased by an order of magnitude vs my last (purely human dev team) build.
I ran into the same issue when building ProForta.com. My solution was to build a companion app that pulls data (including Garmin) via Health Connect. Right now, it seems that every hardware platform from Oura to Garmin is trying to build their moat around user's data. That's not OK - users need access to the combination of all their data to derive meaningful health benefits. Thats the guiding principle behind lifecare.id (https://lifecare.id)
I purchased my Spark back in March before the RAM price increase. My use case is local self-sovereign AI in Healthcare (https://hcvc.net), so I haven't done any gaming. Its been a great driver for Hermes Agent, serving local MoE and dense models up to 122B. It now handles about 90% of my inference load (more complex devops/coding tasks are still routed to cloud). LoRa works well. All my models are open source. There is no NVIDIA dependency in my build. Its also a backend server for my PoC app (https://proforta.com). Zero issues so far. Extremely reliable and gives me straight path to production for VPS deployment.
Very cool PoC for self-sovereign AI! Federated learning across private data resident on user's phone makes sense, but still begs the question of how the user gets access and controls access to their genomic data in a private way. From my perspective, the solution requires giving the user programmatic control over their data, no matter where it resides. At least that's the premise behind work on LifeCare.ID. Let me know if your open to collaborating. https://lifecare.id
I'm building ProForta.com, a prototype app for a self-sovereign AI architecture where you gain control over your health data wherever it resides, the AIs go to your data to gather insights, and raw data custody remains where it was created.
On-device isn't only cheaper/faster - for health data it's the whole point! The self-sovereign community desperately needs an affordable AI appliance and Apple is still the most likely candidate to deliver on the privacy promise.
I got myself a NVIDIA DGX Spark to build on unified-memory hardware at home. Now running a health-AI node for all my longitudinal health data - insights are generated on the box so that raw data never leaks.
the failure wasn’t the science - it was the custody model (your DNA sequence became someone else’s asset). This problem is not limited to 23andMe and the fix is a programatic custody model for health data where the data never leaves your control.
reply