Executive Summary

The Poverty Probability Index (PPI) is a poverty measurement tool for organizations and businesses with a mission to serve the poor. A PPI survey consists of 10 questions about a household’s characteristics using which asset ownership is scored to compute the likelihood that the household is living below the poverty line.

Leveraging Cloud Vision, a field officer would simply have to take a series of photos with their smartphone camera inside and outside of the house and then the Cloud Vision would be able to deduce based on the images the response to the 10 questions by detecting whether the objects mentioned in the survey are present in the household or not.

Requirements

Python
Android Application Development using Kotlin
Some knowledge of Google MLKit
Knowledge of machine learning and computer vision is beneficial

Implementations

Design and Technical Overview

The project has 2 main parts:

The android application which is used to upload data to Cloud Vision using the AutoML API. Cloud Vision then classifies the uploaded image and returns its category. If possible, based on the results from Cloud Vision, the app can automatically fill the needed entries in the PPI Survey form.
The augmentation part: The second part is the data augmentation part. This part is needed because photos household items like washing machines, TVs, etc, in the household environment itself are less found. So images are scraped from google and are augmented using various techniques to increase the size of the dataset which in turns help increase the accuracy of the model trained in using Cloud Vision.

Development

Source Code: https://github.com/openMF/ppi-vision

Testing

Preview link to the application on http://appetize.io :

Latest link: https://appetize.io/app/cqkjoygzdyyyr6fqdfzt44egfy?device=pixel4&osVersion=11.0&scale=75

Older version: https://appetize.io/app/2h371q4enkfa267fx7q60fdn1c?device=nexus5&scale=75&orientation=portrait&osVersion=8.1

Using the above link, the app can be tested on a browser itself without installing it on a physical device.

Other Resources

Meeting Notes

GSoC’22 July 1 Weekly Check-In

GSoC’22 July 8 Weekly Check-In

GSoC’22 July 15 Weekly Check-In

GSoC’22 July 22 Weekly Check-In

GSoC’22 July 29 Weekly Check-In

GSoC’22 August 5 Weekly Check-In

GSoC’22 August 12 Weekly Check-In

GSoC’22 August 19 Weekly Check-In

GSoC’22 August 26 Weekly Check-In

GSoC’22 Sept 2 Weekly Check-In

GSoC’22 Sept 9 Weekly Check-In

Roadmap Planner

maplinks
timeline	true
source	%7B%22title%22%3A%22Roadmap%20Planner%22%2C%22timeline%22%3A%7B%22startDate%22%3A%222022-05-20%2000%3A00%3A00%22%2C%22endDate%22%3A%222022-09-12%2000%3A00%3A00%22%2C%22displayOption%22%3A%22WEEK%22%7D%2C%22lanes%22%3A%5B%7B%22title%22%3A%22Vision%20PPI%22%2C%22color%22%3A%7B%22lane%22%3A%22%233b7fc4%22%2C%22bar%22%3A%22%236c9fd3%22%2C%22text%22%3A%22%23ffffff%22%2C%22count%22%3A1%7D%2C%22bars%22%3A%5B%7B%22title%22%3A%22Feature%201%22%2C%22description%22%3A%22This%20is%20the%20first%20bar.%22%2C%22startDate%22%3A%222018-03-01%2000%3A00%3A00%22%2C%22duration%22%3A8.714285714285714%2C%22rowIndex%22%3A0%2C%22id%22%3A%2299880195-7faf-4b2f-ac44-a38516c4cad5%22%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22title%22%3A%22Feature%202%22%2C%22description%22%3A%22This%20is%20the%20second%20bar.%22%2C%22startDate%22%3A%222018-04-29%2018%3A11%3A20%22%2C%22duration%22%3A4.4226565244708995%2C%22rowIndex%22%3A6%2C%22id%22%3A%22604cae13-314a-4848-b142-95d3570aa3db%22%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A0%2C%22startDate%22%3A%222018-06-18%2022%3A48%3A03%22%2C%22id%22%3A%22739eee1a-3ad0-4154-a8a7-d630942c1b71%22%2C%22title%22%3A%22Feature%203%22%2C%22description%22%3A%22%22%2C%22duration%22%3A6.300469282407407%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A6%2C%22startDate%22%3A%222018-07-12%2005%3A50%3A01%22%2C%22id%22%3A%22d2721513-87bf-4ea5-9981-d3971248c8f5%22%2C%22title%22%3A%22Feature%204%22%2C%22description%22%3A%22%22%2C%22duration%22%3A7.

362615787037037%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A9%2C%22startDate%22%3A%222020

362615787037037%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A8%2C%22startDate%22%3A%222020-05-04%2000%3A00%3A00%22%2C%22id%22%3A%22ce3d00a0-a70a-47fe-a920-f1ffc5efbe30%22%2C%22title%22%3A%22Community%20Bonding%22%2C%22description%22%3A%22%22%2C%22duration%22%3A4.

425742574257426%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A9%2C%22startDate%22%3A%222020

425742574257426%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A8%2C%22startDate%22%3A%222020-06-04%2012%3A49%3A54%22%2C%22id%22%3A%22aea0370f-437f-425a-9f7a-f39023371a87%22%2C%22title%22%3A%22Coding%20Phase%22%2C%22description%22%3A%22%22%2C%22duration%22%3A11.

970297029702971%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A9%2C%22startDate%22%3A%222020

970297029702971%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A8%2C%22startDate%22%3A%222020-08-27%2021%3A08%3A54%22%2C%22id%22%3A%22f876011a-5ee3-426c-a894-6f29673c5fea%22%2C%22title%22%3A%22Final%20Submissions%22%2C%22description%22%3A%22%22%2C%22duration%22%3A1.4158415841584158%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A0%2C%22startDate%22%3A%222020-05-04%2001%3A39%3A48%22%2C%22id%22%3A%22df8e4bfe-f59c-4c6d-8783-429250d4c8fb%22%2C%22title%22%3A%22Work%20on%20getting%20the%20android%20client%20working%22%2C%22description%22%3A%22-%20Update%20all%20dependencies%5Cn-%20Fix%20existing%20issues%20in%20the%20app%5Cn-%20Get%20the%20login%20working%22%2C%22duration%22%3A4.920792079207921%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A6%2C%22startDate%22%3A%222020-06-01%2006%3A39%3A12%22%2C%22id%22%3A%224ddff410-3b62-4b77-b882-894069447b57%22%2C%22title%22%3A%22Review%20all%20existing%20prs%22%2C%22description%22%3A%22%22%2C%22duration%22%3A1.

9207920792079207%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A8%2C%22startDate%22%3A%222020

9207920792079207%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A7%2C%22startDate%22%3A%222020-06-03%2005%3A13%3A39%22%2C%22id%22%3A%227138d5e9-7e0a-4feb-92f7-f2bc889c0f6c%22%2C%22title%22%3A%22Send%20prs%20for%20getting%20the%20app%20work%22%2C%22description%22%3A%22%22%2C%22duration%22%3A1.3267326732673268%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A0%2C%22startDate%22%3A%222020-06-08%2006%3A39%3A12%22%2C%22id%22%3A%223b81b83a-af83-4780-a73e-ecf032e64cab%22%2C%22title%22%3A%22Enhancing%20the%20augmentation%20scripts%20and%20techniques%22%2C%22description%22%3A%22%22%2C%22duration%22%3A1.900990099009901%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A6%2C%22startDate%22%3A%222020-06-15%2009%3A58%3A48%22%2C%22id%22%3A%22f167e7b6-b35a-45d9-9b20-31b0cc0e6591%22%2C%22title%22%3A%22Start%20collecting%2C%20pre-processing%20data%22%2C%22description%22%3A%22%22%2C%22duration%22%3A2.

8613861386138613%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A8%2C%22startDate%22%3A%222020

8613861386138613%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A7%2C%22startDate%22%3A%222020-06-29%2008%3A19%3A00%22%2C%22id%22%3A%224bf93b9f-01c3-4317-9cb3-da2d125024b9%22%2C%22title%22%3A%22Continue%20expanding%20the%20dataset%2C%20training%20models%20and%20improving%20accuracy%22%2C%22description%22%3A%22%22%2C%22duration%22%3A4.900990099009901%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A0%2C%22startDate%22%3A%222020-06-29%2008%3A19%3A00%22%2C%22id%22%3A%2297e6d31a-2a5e-4e8e-bc13-4d704724bc1b%22%2C%22title%22%3A%22Do%20work%20needed%20for%20integrating%20the%20AutoML%20API%22%2C%22description%22%3A%22Add%20option%20to%20click%20and%20upload%20photos%20and%20integrate%20the%20AutoML%20API%22%2C%22duration%22%3A2.891089108910891%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A6%2C%22startDate%22%3A%222020-07-13%2006%3A39%3A12%22%2C%22id%22%3A%2240c99c0f-9f2d-4c9f-a65b-9636755387b8%22%2C%22title%22%3A%22Test%20models%20via%20the%20app%22%2C%22description%22%3A%22Test%20the%20models%20in%20real%20time%20via%20the%20android%20application%20and%20continue%20to%20improve%20the%20accuracy%20as%20needed%20by%20expanding%20the%20datasets%20and%20other%20methods.%22%2C%22duration%22%3A2.9306930693069306%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A0%2C%22startDate%22%3A%222020-08-03%2013%3A18%3A24%22%2C%22id%22%3A%22e4032d86-a976-4e55-b003-70bbcbd8cbd9%22%2C%22title%22%3A%22Enhance%20the%20UI%2FUX%20of%20the%20android%20app%22%2C%22description%22%3A%22%22%2C%22duration%22%3A2.8613861386138613%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A6%2C%22startDate%22%3A%222020-08-05%2015%3A12%3A28%22%2C%22id%22%3A%22792d7d8f-dcea-4c19-b7f0-de7e60c44c5a%22%2C%22title%22%3A%22Add%20new%20features%20to%20the%20app%22%2C%22description%22%3A%22-%20About%20section%5Cn-%20Shift%20to%20a%20bottom%20nav%20bar%5Cn-%20Follow%20proper%20code%20conventions%5Cn-%20Improve%20the%20code%20style%20of%20the%20application%20as%20it%20is%20not%20good%20right%20now.%22%2C%22duration%22%3A2.

198019801980198%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A8%2C%22startDate%22%3A%222020

198019801980198%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A7%2C%22startDate%22%3A%222020-08-17%2011%3A38%3A36%22%2C%22id%22%3A%22b51f0da0-ec9b-4dcf-b163-c59e53062b19%22%2C%22title%22%3A%22Documentation%22%2C%22description%22%3A%22-%20Document%20all%20the%20work%20done%5Cn-%20Update%20the%20project%20Wiki%20and%20Confluence%20page%5Cn-%20Create%20a%20detailed%20README%20for%20the%20GitHub%20repo%22%2C%22duration%22%3A1.8712871287128714%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A2%2C%22startDate%22%3A%222020-08-24%2011%3A38%3A36%22%2C%22id%22%3A%22f02cc7c7-e97a-485a-9674-b9ab0387e6ab%22%2C%22title%22%3A%22Submit%20for%20final%20evaluations%22%2C%22description%22%3A%22%22%2C%22duration%22%3A1.8712871287128714%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A2%2C%22startDate%22%3A%222020-06-22%2008%3A19%3A00%22%2C%22id%22%3A%2255ab381b-f1df-48da-a40b-f3465ea78b19%22%2C%22title%22%3A%22Understand%20how%20the%20AutoML%20backend%20works%20for%20classifying%20images%22%2C%22description%22%3A%22-%20Make%20my%20own%20CNN%20for%20the%20purpose%20of%20image%20classification%20to%20understand%20the%20process%20and%20add%20it%20to%20the%20project%20repo%20so%20others%20can%20understand%20it%20too.%22%2C%22duration%22%3A2.891089108910891%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A2%2C%22startDate%22%3A%222020-07-13%2014%3A58%3A13%22%2C%22id%22%3A%22559fc463-5bbc-4a69-8fc9-169758acb9d2%22%2C%22title%22%3A%22Add%20functionality%20for%20auto-filling%20the%20survey%22%2C%22description%22%3A%22%22%2C%22duration%22%3A2.

8415841584158414%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A7%2C%22startDate%22%3A%222022

8415841584158414%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A6%2C%22startDate%22%3A%222022-05-

21%2018%3A03%3A33%22%2C%22id%22%3A%2220f27d30

23%2000%3A00%3A00%22%2C%22id%22%3A%2220f27d30-1058-4bcc-ba4e-70873fa8b5e2%22%2C%22title%22%3A%22Community%20Bonding%22%2C%22description%22%3A%22Community%20Bonding%22%2C%22duration%22%3A3.

1782178217821784%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A7%2C%22startDate%22%3A%222022

01980198019802%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A6%2C%22startDate%22%3A%222022-06-

14%2002%3A36%3A49%22%2C%22id%22%3A%22337f529f

14%2004%3A16%3A38%22%2C%22id%22%3A%22337f529f-1293-4c5c-835e-

44fea9e8a7e0%22%2C%22title%22%3A%22Coding%20Phase%22%2C%22description%22%3A%22%22%2C%22duration%22%3A10

44fea9e8a7e0%22%2C%22title%22%3A%22Coding%20Phase%22%2C%22description%22%3A%22%22%2C%22duration%22%3A11.

811881188118813%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A7%2C%22startDate%22%3A%222022

188118811881187%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A6%2C%22startDate%22%3A%222022-09-

01%2004%3A30%3A53%22%2C%22id%22%3A%2244dc00d1

01%2009%3A30%3A18%22%2C%22id%22%3A%2244dc00d1-e55d-4aa7-ac3b-62409144fadc%22%2C%22title%22%3A%22Final%20Submissions%22%2C%22description%22%3A%22%22%2C%22duration%22%3A1.

2871287128712872%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A2%2C%22startDate%22%3A%222022

4752475247524752%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A2%2C%22startDate%22%3A%222022-06-19%2020%3A40%3A23%22%2C%22id%22%3A%22d96ca194-f282-40e0-8d2d-cef01dee4e86%22%2C%22title%22%3A%22Start%20collecting%2C%20pre-processing%20data%22%2C%22description%22%3A%22%22%2C%22duration%22%3A3.4554455445544554%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A2%2C%22startDate%22%3A%222022-07-18%2004%3A59%3A24%22%2C%22id%22%3A%22aebd99a8-2bcd-4022-8665-1c102811b487%22%2C%22title%22%3A%22Test%20models%20via%20the%20app%22%2C%22description%22%3A%22%22%2C%22duration%22%3A1.8415841584158417%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A1%2C%22startDate%22%3A%222022-07-03%2020%3A40%3A23%22%2C%22id%22%3A%22d49e789c-d7ba-4ec2-94bc-57bd56a7de48%22%2C%22title%22%3A%22Add%20new%20features%20to%20the%20app%22%2C%22description%22%3A%22%22%2C%22duration%22%3A3%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A4%2C%22startDate%22%3A%222022-07-14%2001%3A11%3A17%22%2C%22id%22%3A%22d8048117-9624-40fc-a0ad-22551db5e626%22%2C%22title%22%3A%22Publishing%20The%20Changes%22%2C%22description%22%3A%22%22%2C%22duration%22%3A2.

4257425742574257%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A5%2C%22startDate%22%3A%222022

5742574257425743%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A4%2C%22startDate%22%3A%222022-08-

29%2001%3A39%3A48%22%2C%22id%22%3A%229272b667

26%2012%3A07%3A07%22%2C%22id%22%3A%229272b667-016a-4b62-9a7c-

9be4d38251b0%22%2C%22title%22%3A%22Documentations%22%2C%22description%22%3A%22%22%2C%22duration%22%3A1

9be4d38251b0%22%2C%22title%22%3A%22Documentations%22%2C%22description%22%3A%22%22%2C%22duration%22%3A2.

9603960396039604%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A3%2C%22startDate%22%3A%222022

128712871287129%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A3%2C%22startDate%22%3A%222022-07-25%2001%3A39%3A48%22%2C%22id%22%3A%22d43c6e18-1fb4-4ecf-8d48-bf71110d0d1b%22%2C%22title%22%3A%22Enhance%20the%20UI%2F%20UX%20of%20the%20android%20app%22%2C%22description%22%3A%22%22%2C%22duration%22%3A3.2475247524752477%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A3%2C%22startDate%22%3A%222022-08-17%2020%3A11%3A52%22%2C%22id%22%3A%22fe2a0fc6-2243-4d65-8749-

d54c4d6ef81f%22%2C%22title%22%3A%22Publishing%20The%20Changes%22%2C%22description%22%3A%22%22%2C%22duration%22%3A3%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A6%2C%22startDate%22%3A%222022

d54c4d6ef81f%22%2C%22title%22%3A%22Publishing%20The%20Changes%22%2C%22description%22%3A%22%22%2C%22duration%22%3A3%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A5%2C%22startDate%22%3A%222022-07-20%2021%3A51%3A41%22%2C%22id%22%3A%229124bba7-79e3-47c6-b746-6dbdfff6843d%22%2C%22title%22%3A%22Mid%20Term%20Evaluation%22%2C%22description%22%3A%22%22%2C%22duration%22%3A1.4554455445544554%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A2%2C%22startDate%22%3A%222022-08-01%2011%3A38%3A37%22%2C%22id%22%3A%2219faf685-586c-4295-ba90-c1193f60b4a7%22%2C%22title%22%3A%22Integrating%20new%20ML%20model%22%2C%22description%22%3A%22%22%2C%22duration%22%3A4.435643564356436%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A1%2C%22startDate%22%3A%222022-08-02%2012%3A35%3A38%22%2C%22id%22%3A%22f37e085e-ca05-4873-9ed3-bb88dfe51811%22%2C%22title%22%3A%22Adding%20the%20PPI%20Fragments%22%2C%22description%22%3A%22%22%2C%22duration%22%3A2.772277227722772%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A1%2C%22startDate%22%3A%222022-06-06%2001%3A39%3A48%22%2C%22id%22%3A%2274c3c737-fe75-4884-96bb-a3423ca32db6%22%2C%22title%22%3A%22Fixing%20the%20API%20Endpoint%20Issues%22%2C%22description%22%3A%22%22%2C%22duration%22%3A3.277227722772277%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A0%2C%22startDate%22%3A%222022-05-20%2023%3A45%3A44%22%2C%22id%22%3A%2277734695-5bf8-4739-92ef-3d65595fea6c%22%2C%22title%22%3A%22Understanding%20about%20Login%20System%20working%22%2C%22description%22%3A%22%22%2C%22duration%22%3A3.2475247524752477%2C%22pageLink%22%3A%7B%7D%7D%2C%7B%22rowIndex%22%3A4%2C%22startDate%22%3A%222022-06-24%2015%3A26%3A44%22%2C%22id%22%3A%22d64a4f5c-3421-4cd1-9ed0-b899a3c1c9bc%22%2C%22title%22%3A%22Updating%20the%20color%20schemes%22%2C%22description%22%3A%22%22%2C%22duration%22%3A2.504950495049505%2C%22pageLink%22%3A%7B%7D%7D%5D%7D%5D%2C%22markers%22%3A%5B%7B%22title%22%3A%22Marker%201%22%2C%22markerDate%22%3A%222018-03-15%2000%3A00%3A00%22%7D%2C%7B%22markerDate%22%3A%222020-09-

06%2022%3A20%3A11%22%2C%22title%22%3A%22We%20are%20here%22%7D%5D%7D

hash
06%2022%3A20%3A11%22%2C%22title%22%3A%22We%20are%20here%22%7D%2C%7B%22markerDate%22%3A%222022-09-11%2020%3A40%3A23%22%2C%22title%22%3A%22We%20are%20here%22%7D%5D%7D
pagelinks
title	Roadmap%20Planner

a000f17f1efce73cf77e0299132db2ca