Introducing Nutrify
Short Version
Nutrify is an iOS application designed to make learning about whole foods fun and in turn, reduce the intake of ultra-processed foods.
It works on the premise of “take a photo of food and learn about it”.
As in, it uses computer vision to identify different whole foods (we focus on whole foods rather than barcodes) and displays relevant nutrition and storage information about them.
Nutrify is for anyone who wants to learn more about foods as well as about their own food habits and trends.
Due to its fun and interactive design, Nutrify can also be an education tool for teaching kids or students about food.
- See the launch on Product Hunt.
- Watch the launch video on YouTube.
- Download Nutrify on the iOS App Store.
Take a photo of food and learn about it. Download Nutrify today on the Apple App Store.
Longer Version
It’s here!
The app my brother and I have been dreaming about for years.
And we’ve built it.
In 2017, I started learning machine learning/AI with the explicit goal of “Combine AI with my knowledge of fitness and nutrition to help the world move more and eat better.”
The initial goals I wrote in 2017 when I started learning AI and machine learning with my self-created AI Masters Degree.
Nutrify is the nutrition part of that goal.
It’s designed to be an easy and fun way to learn about whole foods.
I say whole foods on purpose.
Because Nutrify focuses on foods you could forage for or farm or make in your kitchen.
As in, foods without barcodes or packaging.
If Nutrify has one central dogma, it’s that.
We believe whole foods are healthier than ultra-processed foods.
When I studied nutrition and food science at university, that was the central point.
Basically any diet other than the Standard American Diet (SAD for short, funny), a diet full of ultra-processed foods, is a good alternative (AKA diets full of whole foods).
The Standard American Diet (SAD for short). If I learned anything in a nutrition and food science degree it's that nutrition can be summed up by avoiding foods like this. And that most of food science is dedicated to making foods like this.
There were also many fun and interesting things I’d learn about food but seem to forget whenever I actually wanted to remember them.
In other words, I’d remember the information for passing exams but not when I was looking at food or eating it.
When I learned about computer vision I had the idea of bringing this knowledge to the food.
As in, use the camera as the intelligence layer.
This is of course, not new.
Computer vision has been used for many different use cases. Even for identifying foods in other apps.
Nutrify’s differentiating factor is that it focuses on food categories and discovery rather than counting calories.
Because if you’re eating whole foods, moving often and sleeping well, you don’t need to count calories.
To keep it simple, I love food and I love machine learning.
Nutrify is the love child of these two passions.
Who’s it for?
Nutrify is for people who would like to learn more about food.
Either more about foods you’re familiar with or foods you’ve never tried before.
It can be used as a food discovery tool.
Or as a way to track trends and eating habits over time (see the stats and trends feature below).
One use case we’ve seen is parents teaching their kids about different foods at the supermarket or in the kitchen by taking photos of them and having Nutrify read out nutrition summaries.
Key features and benefits
📸->🍍 FoodVision AI
Nutrify is powered by a computer vision model capable of identifying 420 foods (for now, more coming soon).
Simply point your phone at food and take a photo, Nutrify will do its best to identify what’s in it and show relevant information.
All image processing happens on-device using the Neural Engine, so predictions happen really fast and work offline.
Nutrify's computer vision models run on-device and leverage the Neural Engine (Apple's chip for machine learning) so they happen fast and work without an internet connection (even on the top of a mountain). The video is a live screen recording of a lasagne lunch hike break on top of Mt. Tamborine, Queensland, Australia.
📔 Create a Visual Food Diary
When you take a photo with Nutrify, you have the option to save or delete it.
Saving it stores it in your Visual Food Diary, an excellent way to reflect on what you’ve eaten.
Don't count calories. Look at what you've eaten instead. Nutrify enables an easy way to keep a visual log of what you've eaten. Something to reflect on or show a healthcare practitioner.
In my experience with keeping food diaries, I’ve found that numbers on a page (calorie counting) are too specific for me.
Instead, I much prefer seeing what I’ve eaten (ideally, a wide range of whole foods).
You could use this to show a nutrition coach or wellness practitioner what you’ve eaten and get their insights.
🔍 Discover new foods and complete the Nutridex
I loved playing Pokémon as a kid.
I used to think the Pokédex was magic.
So my brother and I made a big effort to make Nutrify feel the same.
For version 1.0, the Nutridex contains information 420 different foods across 22 different categories.
Nutrify contains information for 420 foods from 22 different categories including fruit, vegetables, nuts and seeds, legumes, meat, fungi and many more.
Each comes with a beautifully designed custom food icon crafted by our skilful designer (thank you Grace!).
And the first time you take a photo of a food you haven’t seen before, a delightful unlock sound will play and it’ll be added to your Nutridex (see the launch video for what this looks like). A small incentive to continually try new foods.
The Nutridex contains information about different foods and displays which foods you've discovered/haven't discovered with the camera.
The Nutridex will be constantly growing over time.
So completing it may actually be a continually moving target.
As of writing, my Nutridex is at 357/420.
If you find that your favourite food isn’t in the Nutridex, let us know so we can expand it!
📊 Get stats and trends of your food habits over time
After you’ve used Nutrify for a while, stats and trends will automatically populate the Summary tab based on your Visual Food Diary.
This includes most eaten foods, most eaten categories, most common breakfast, lunch and dinner foods and more.
You can use these statistics and trends to gain insights on your food habits and adjust if necessary.
As you take photos with Nutrify, the Summary tab (paid feature) will populate automatically with stats and trends. You'll also be able to see how different foods affect your heart rate as well as set goals for consuming/avoiding certain foods/categories.
We’ll be adding more analytics and insights over time as well so feel free to suggest something you’d like to see.
New: You can now also use the Summary tab to see how different foods affect your heart rate (requires heart rate data in Apple Health) and set goals for consuming or avoiding different foods/categories.
How does it work?
Nutrify is a combination of machine learning, iOS, a database and design.
It's bootstrapped in a converted lounge room office in Brisbane, Australia by myself and my brother Josh working together part-time.
A brief overview of how Nutrify works. One missing piece is the feedback loop that happens when Nutrify gets something wrong. We use this information to improve future models. Like a data flywheel.
Machine Learning (Daniel)
Computer vision models are fine-tuned versions of timm (PyTorch Image Models) models from Hugging Face (thank you to Ross Wightman for the incredible work) on a custom dataset.
All training happens locally on a single GPU (one GPU, one dream).
Experiment and artifact tracking happens all with Weights and Biases.
We're currently at 1000+ models trained as well as 100+ versions of datasets/labels.
Food images are collected from open-source datasets as well as manually collected (we've manually taken 50,000+ images of food/not food - never underestimate the power of manual data collection).
We use an iterative training loop similar to Tesla’s data flywheel but for food images. You can see this setup explained on YouTube.
iOS (Josh)
All of Nutrify is built in SwiftUI so it runs and feels like a native iOS app.
All machine learning models run on-device using Core ML and perform inference in under 10ms (no internet connection required + very fast!).
We leverage several of Apple's in-built APIs to ensure Nutrify runs as clean as possible (this is why it requires iOS 16+).
Database
Our nutrition data comes from open-source nutrition databases (FDC and FSANZ).
Metadata, object storage and authentication (sign-in with Apple etc) happens via Supabase.
Food Icons (Grace)
All food icons are custom-designed specifically for Nutrify.
Pricing
Nutrify is free to use for up to 20 images without an account and unlimited images with an account.
This means you could complete the Nutridex for free by signing up for an account and taking photos of food.
Personalised food stats, summaries and trends are a paid feature.
Our pricing is monthly with a discount for yearly.
- $3.99USD/month ($47.90USD/year)
- $24.99USD/year (47% cheaper than monthly)
Please note these are base prices and actual prices may differ in the Apple App Store depending on the region. This is at Apple’s discretion and out of our control.
Where it’s going
Nutrify version 1.0 just launched but we’re already thinking about the future.
Some things we’re thinking about:
- Nutrify is built in Australia - There is a focus on whole foods available to us but we’ll expand this over time. However, many of the existing 420 foods are common globally. We’d like to take this to 1000+ by the end of the year.
- Our computer vision models aren’t perfect - Nutrify will make mistakes. But that’s half the fun! The good news is, it’s easy to correct by tapping another option in the swap bar or by tapping the pencil to add a custom food (the custom foods will also let us know which foods to add to the Nutridex).
- Multiple foods, dishes and complex meals - For now, Nutrify focuses on the premise of one photo, one food. But as we upgrade the computer vision models, they’ll be able to handle multiple foods and dishes better. Our advice is to take photos of food when you’re making it and enjoy it when it’s made.
- Other potential ideas - Food information for specific demographics, for example, children, newborn baby food tracking (e.g. “what foods has my baby tried?”), parents looking to conceive and more. We’ve tried to stay away from specifically recommending certain foods (other than whole foods) so this is an area which may require a bit more research.
Contact and learn more
If you have any ideas or foods you’d like to see in Nutrify, feel free to let us know!
Our contact email is [email protected].
And you can find us elsewhere:
In the meantime, have fun learning about food!
– Daniel and Josh
FAQ
What is Nutrify?
Nutrify is an iOS app that uses computer vision to identify whole foods from photos and provides relevant nutrition and storage information. It aims to make learning about food fun and informative.
How does Nutrify work?
By simply taking a photo of whole food, Nutrify uses a computer vision model to identify the food and then displays relevant nutrition information. The app focuses on whole foods rather than barcodes or foods in packaging.
Who can benefit from using Nutrify?
Anyone interested in learning more about food, tracking their food habits, discovering new foods, or using it as an educational tool to learn about food and nutrition.
What features does Nutrify offer?
Nutrify's main features include FoodVision AI for identifying foods, a Visual Food Diary for tracking what you eat, the Nutridex for discovering new foods, and stats and trends analysis over time.
Does Nutrify require an internet connection to work?
No, all image processing happens on-device using the Neural Engine, allowing for fast predictions and offline functionality.
How many foods can Nutrify identify?
Currently, Nutrify is capable of identifying 420 different whole foods.
Is Nutrify free to use?
Nutrify can be used to identify and learn about foods for free for up to 20 images without an account, and unlimited images with an account. Personalized food stats and trends are a paid feature for $3.99USD/month or $24.99/year.
How accurate is Nutrify’s food identification?
While Nutrify can identify many foods very well, our computer vision models aren’t perfect. They will make mistakes. Good news is, you can easily correct errors by selecting another option or adding a custom food.
Can Nutrify identify multiple foods in one photo?
Currently, Nutrify focuses on identifying one food item per photo, but future updates aim to improve its capability to handle multiple foods and complex meals better.
What sets Nutrify apart from other food tracking apps?
Nutrify emphasizes food discovery and learning over calorie counting, offering a fun and interactive way to explore nutrition information for whole foods.
How can I download Nutrify?
Nutrify is available for download on the iOS App Store. You can find it by searching for "Nutrify".
What are the future plans for Nutrify?
We’d love to continue to expand the food database, improve our computer vision models for better accuracy, and exploring additional features such as demographic-specific food information and tracking for specific needs like “foods my baby has eaten” or specific dietary preferences.