2017 Winner & Scale Up Runner 2018Farm.ink: Analysing livestock social media data for farmer chatbot
The Inspire Challenge is an initiative to challenge partners, universities, and others to use CGIAR data to create innovative pilot projects that will scale. We look for novel approaches that democratize data-driven insights to inform local, national, regional, and global policies and applications in agriculture and food security in real time; helping people–especially smallholder farmers and producers–to lead happier and healthier lives.
This proposal was selected as a 2017 winner, with the team receiving 100,000 USD to put their ideas into practice. The team came runners up for the Scale Up award the following year, receiving an additional USD 125,000 for their outstanding ability to demonstrate the project’s proven viability and potential for impact.
Farm.ink: Analysing livestock social media data for farmer chatbot
In 2017, Farm.ink was awarded Big Data’s Inspire Challenge start up grant of US$100,000. Farm.ink’s ambition was to build a machine-learning classifier that would turn the hundreds of thousands of social media posts relating to livestock in Kenya into research insights about livestock health in the country. Farm.ink would then combine these insights with International Livestock Research Institute (ILRI) data to deliver timely, targeted information to small-scale dairy farmers in East Africa through their mobile phones.
Over the course of the grant, Farm.ink created the Africa Farmers Club: a Facebook community which now has over 120,000 members. The Farm.ink team also built a chatbot on the Messenger platform to deliver dairy farmers information on productivity, markets and livestock management. A survey by the project found that 92% of dairy farmers reported changing their farming practices based on information received through this service.
During Big Data’s conference in Nairobi, Kenya in October 2018, Farm.ink received an additional scale-up grant of US$125,000 that is going towards scaling Farm.ink’s digital services up and out across Africa. In partnership with IDEO.org, the team is now focusing on building a web platform to enable farmers, across multiple value chains, to share questions and advice and access expert agronomic content. Alongside dairy, Farm.ink is expanding to maize (with a focus on the recent outbreak of Fall Armyworm) and other common horticulture crops such as tomato. They aim to reach over 1 million monthly active users in Africa within 2 years.
Step by step
The project was one of five winners of the Inspire Challenge 2017 and was awarded US$100K at the inaugural annual convention of the CGIAR Platform Big Data in Agriculture, during 19-22 of September.
Building a natural language processing (NLP) classifier
The team successfully built a NLP classifier to accurately label incoming data. Farm.ink completed multiple workshops with the ILRI team to create a labelling protocol for the data. Using this protocol, they successfully analysed hundreds of thousands of posts, comments and images generated by Kenyan farmers and labelled over 26,000 rows of data using a human-in-the-loop labelling method. They then built and trained an NLP based classifier to label livestock posts (e.g. health issue, buying/selling). This work allowed to shine a light on the principal problems and concerns of livestock farmers in Kenya, which can be analysed at various levels. For instance, the data revealed that 40% of the information shared relates to the buying and selling cattle. Drilling down one level and looking into the animal husbandry category, the most popular topic of discussion is feeds.
Developing analytical tools to turn the data into actionable information
The team has built open-source analytical tools to turn the data into actionable information for dairy farmers and scientific researchers. Using the labelled data the team created a dashboard tool to analyse trends in the data that the ILRI team and others can access. The codebase is built using open-source tools. As planned, this will be made publicly accessible along with the labelled data by the end of 2018, enabling others to build on the results.
Creating a dairy chatbot service
The team designed a simple and engaging way for farmers to receive this information on their phones. Over the last year the team has prototyped and tested multiple chatbot features for livestock farmers using the labelled data. They have combined actionable information from the social feed data with ILRI data to create targeted farmer alerts through the chatbot. These alerts include information on animal health issues, an analysis of local milk prices and a report on local cows for sale.
Measuring the results
The team has seen real results on the ground: Through the project brand, Africa Farmers Club, the userbase has now reached 24,000 dairy farmers and is on track to exceed the target of 40,000 by the end of 2018. A recent survey of 406 dairy chatbot users found that 92% reported having changed the way they farm based on information received through our services.
US$125K scale-up grant
The project was a runner up in the Inspire Challenge Scale Up 2018 and was awarded US$125K at the second annual convention of the CGIAR Platform for Big Data in Agriculture, during 3-5 of October.
In the initial Inspire phase, the Farm.Ink team built an active online community (Africa Farmers Club) of over 100,000 farmers and used machine-learning classifiers to turn this unstructured social feed into actionable insights. They combined these insights with content and data from ILRI to deliver information back to livestock farmers through a chatbot. Their plan for 2019 is to expand on this considerably. They are in the process of building a web platform that enables farmers, across multiple value chains, to share questions and advice and access expert agronomic content. Alongside dairy, they are expanding to include maize (with a particular focus on the recent outbreak of Fall Armyworm) as well as common horticulture crops such as tomato.