2017 WinnerFarm.ink: Analysing livestock social media data for farmer chatbot
The Inspire Challenge is an initiative to challenge partners, universities, and others to use CGIAR data to create innovative pilot projects that will scale. We look for novel approaches that democratize data-driven insights to inform local, national, regional, and global policies and applications in agriculture and food security in real time; helping people–especially smallholder farmers and producers–to lead happier and healthier lives.
This proposal was selected as a 2017 winner, with the team receiving 100,000 USD to put their ideas into practice. The team came runners up for the Scale Up award the following year, receiving an additional USD 125,000 for their outstanding ability to demonstrate the project’s proven viability and potential for impact.
Farm.ink: Analysing livestock social media data for farmer chatbot
The largest farming Facebook groups are all located in sub-Saharan Africa and the combined membership of the top four groups spans 100,000s of individuals. Inside these groups we find 10,000s of historical posts on the topic of dairy farming & livestock. These posts often contain detailed reports of livestock disease as well as queries and comments about productivity and management concerns. As yet, these posts remain unanalyzed. In 2017 ILRI launched Community Disease Reporting (CDR): a programme training animal health attendants to report livestock disease in their communities via their mobile phones. Focused on Northern Kenya, this system provides ILRI with real-time livestock disease updates in lower-income, rural areas. ILRI is also engaged with 1,000s of livestock farming households and has 100s of enumerators on the ground embedded within communities. Farmers and enumerators are communicating health and productivity data via mobile systems.
The project proposes to combine social media data with ILRI data to create an open-source platform that provides small-scale dairy farmers in East Africa with targeted, timely information to enable them to make better decisions on their farm.
Farm.ink currently broadcasts messages to thousands of farmers, including many livestock farmers, through its chatbot product, which is built on the Facebook Messenger platform and sends a tailored daily feed of farming news to users. Information on diseases and other alerts will be added to this daily digest service. With over 40% of livestock bot users engaging with the service every day, the chatbot is a very efficient channel to broadcast information that farmers need to protect their livestock.
Step by step
The project was one of five winners of the Inspire Challenge 2017 and was awarded US$100K at the inaugural annual convention of the CGIAR Platform Big Data in Agriculture, during 19-22 of September.
Building a natural language processing (NLP) classifier
The team successfully built a NLP classifier to accurately label incoming data. Farm.ink completed multiple workshops with the ILRI team to create a labelling protocol for the data. Using this protocol, they successfully analysed hundreds of thousands of posts, comments and images generated by Kenyan farmers and labelled over 26,000 rows of data using a human-in-the-loop labelling method. They then built and trained an NLP based classifier to label livestock posts (e.g. health issue, buying/selling). This work allowed to shine a light on the principal problems and concerns of livestock farmers in Kenya, which can be analysed at various levels. For instance, the data revealed that 40% of the information shared relates to the buying and selling cattle. Drilling down one level and looking into the animal husbandry category, the most popular topic of discussion is feeds.
Developing analytical tools to turn the data into actionable information
The team has built open-source analytical tools to turn the data into actionable information for dairy farmers and scientific researchers. Using the labelled data the team created a dashboard tool to analyse trends in the data that the ILRI team and others can access. The codebase is built using open-source tools. As planned, this will be made publicly accessible along with the labelled data by the end of 2018, enabling others to build on the results.
Creating a dairy chatbot service
The team designed a simple and engaging way for farmers to receive this information on their phones. Over the last year the team has prototyped and tested multiple chatbot features for livestock farmers using the labelled data. They have combined actionable information from the social feed data with ILRI data to create targeted farmer alerts through the chatbot. These alerts include information on animal health issues, an analysis of local milk prices and a report on local cows for sale.
Measuring the results
The team has seen real results on the ground: Through the project brand, Africa Farmers Club, the userbase has now reached 24,000 dairy farmers and is on track to exceed the target of 40,000 by the end of 2018. A recent survey of 406 dairy chatbot users found that 92% reported having changed the way they farm based on information received through our services.
US$125K scale-up grant
The project was a runner up in the Inspire Challenge Scale Up 2018 and was awarded US$125K at the second annual convention of the CGIAR Platform for Big Data in Agriculture, during 3-5 of October.
Scaling up and out: Towards 1 million active users
The team estimates that there are over 100m posts and comments relating to farming across social channels and over 20m farmers in Africa with a smartphone and mobile data connection. Farmers are talking online at massive scale, and both farmers and researchers should be able to make use of this fact! The project has had incredible success building an active farmer community, and if we can move from 100,000s of social farming data points to 100s of millions this can become one of the most valuable data inputs to the project itself and beyond. The method can also be replicated for other value chains such as maize and horticulture. The effective collaboration between Farm.ink and ILRI can be replicated across other institutions and value chains. We propose a minimum of 4 new value chains to strategically focus on. The objective would be to scale the digital services to over 1 million monthly active users in Africa within 2 years.
Establishing the partnerships needed to reaching these big ambitions
Stay tuned for more updates!
Meet all the 2017 Winners