2017 Winner & Scale Up Runner 2018Farm.ink: Analysing livestock social media data for farmer chatbot
The Inspire Challenge is an initiative to challenge partners, universities, and others to use CGIAR data to create innovative pilot projects that will scale. We look for novel approaches that democratize data-driven insights to inform local, national, regional, and global policies and applications in agriculture and food security in real time; helping people–especially smallholder farmers and producers–to lead happier and healthier lives.
This proposal was selected as a 2017 winner, with the team receiving 100,000 USD to put their ideas into practice. The team came runners up for the Scale Up award the following year, receiving an additional USD 125,000 for their outstanding ability to demonstrate the project’s proven viability and potential for impact.
Farm.ink: Analysing livestock social media data for farmer chatbot
Smallholder farmers in Kenya produce 60 percent of the country’s food, yet they face a variety of challenges—lack of access to vital information being one of the most pressing. As a result of the information gap, Kenya produces less than 30 percent of what’s required to feed the nation.
In order to improve Kenyan production, Farm.ink seeks to connect geographically isolated farmers and provide research insights about livestock health. After winning a 2017 Inspire Challenge start up grant, the project created a digital community, the Africa Farmers Club, a Facebook group which now has over 12,00 members. For the first time, Kenyan farmers have access to information that can improve their farming practices via social networks.
Additionally, the Farm.ink team built a chatbot on the Facebook Messenger platform to deliver dairy farmers information on productivity, markets, and livestock management. A survey by the project found that 92 percent of dairy farmers reported changing their farming practices based on information received through this service.
Moving forward with funds from the 2018 Inspire Challenge scale up grant, the project will expand their digital services up and out across Africa. In partnership with IDEO.org, the team will build a web platform to enable farmers across multiple value chains, such as maize and tomato, to ask questions, share advice, and access expert agronomic content.
The project also seeks to build a machine-learning classifier that would turn the hundreds of thousands of Africa Farmers Club posts relating to livestock in Kenya into research insights about livestock health in the country. Farm.ink will combine these insights with International Livestock Research Institute (ILRI) data to deliver timely, targeted information to small-scale dairy farmers in East Africa through their mobile phones.
Step by step
The project was one of five winners of the Inspire Challenge 2017 and was awarded US$100K at the inaugural annual convention of the CGIAR Platform Big Data in Agriculture, 19-22 September 2017.
Building a natural language processing (NLP) classifier
The team built a NLP classifier to accurately label incoming data. Farm.ink completed multiple workshops with the ILRI team to create a data labeling protocol. Using this protocol, hundreds of thousands of posts, comments and images generated by Kenyan farmers were analysed and labelled over 26,000 rows of data using a human-in-the-loop labeling method. They then built and trained a NLP-based classifier to label livestock posts (e.g. health issue, buying/selling).
This work brought to light various problems and concerns of livestock farmers in Kenya. For instance, the data revealed that 40 percent of the information shared relates to the buying and selling of cattle. Drilling down one level and looking into the animal husbandry category, the most popular topic of discussion is feeds.
Developing analytical tools to turn the data into actionable information
The team built open-source analytical tools to turn the data into actionable information for dairy farmers and scientific researchers. Using the labelled data, the team created a dashboard tool to analyse data trends that the ILRI team and others can access. The tool, along with the labelled data, will be publicly accessible by the end of 2018, enabling others to build on the results.
Creating a dairy chatbot service
The team designed a simple and engaging way for farmers to receive information on their phones. Over the last year, the team has prototyped and tested multiple chatbot features for livestock farmers using the labelled data. They have combined actionable information from the social feed data with ILRI data to create targeted farmer alerts through the chatbot. These alerts include information on animal health issues, an analysis of local milk prices, and a report on local cows for sale.
Real results on the ground
The userbase of the Africa Farmers Club has reached 24,000 dairy farmers and is on track to exceed 40,000 by the end of 2018. A recent survey of 406 dairy chatbot users found that 92 percent reported having changed the way they farm based on information received through our services.
US$125K scale-up grant
The project was a runner up in the Inspire Challenge Scale Up 2018 and was awarded US$125K at the second annual convention of the CGIAR Platform for Big Data in Agriculture, 3-5 October 2018.
Understanding the key needs of digital farmers
Farm.ink worked in deep collaboration with partner IDEO.org to understand the key needs of digital farmers. In-depth interviews were conducted with 28 farmers, including those who had used Farm.ink services before and those who had not.
The key needs identified from conversations were:
- a better theoretical foundation for farming
- social safe space, as compared with current online forums
- more practical farming education
These user-generated themes were used to begin prototyping web platform concepts.
In the initial Inspire phase, the Farm.Ink team built an active online community (Africa Farmers Club) of over 100,000 farmers and used machine-learning classifiers to turn this unstructured social feed into actionable insights. They combined these insights with content and data from ILRI to deliver information back to livestock farmers through a chatbot. Their plan for 2019 is to expand on this considerably. They are in the process of building a web platform that enables farmers, across multiple value chains, to share questions and advice and access expert agronomic content. Alongside dairy, they are expanding to include maize (with a particular focus on the recent outbreak of Fall Armyworm) as well as common horticulture crops such as tomato.