2017 Winner & Scale Up Runner 2018Farm.ink: Analysing livestock social media data for farmer chatbot
The Inspire Challenge is an initiative to challenge partners, universities, and others to use CGIAR data to create innovative pilot projects that will scale. We look for novel approaches that democratize data-driven insights to inform local, national, regional, and global policies and applications in agriculture and food security in real time; helping people–especially smallholder farmers and producers–to lead happier and healthier lives.
This proposal was selected as a 2017 winner, with the team receiving 100,000 USD to put their ideas into practice. The team came runners up for the Scale Up award the following year, receiving an additional USD 125,000 for their outstanding ability to demonstrate the project’s proven viability and potential for impact.
Farm.ink: Analysing livestock social media data for farmer chatbot
Smallholder farmers in Kenya produce 60 percent of the country’s food, yet they face a variety of challenges—lack of access to vital information being one of the most pressing. As a result of the information gap, Kenya produces less than 30 percent of what’s required to feed the nation.
In order to improve Kenyan production, Farm.ink seeks to connect geographically isolated farmers and provide research insights about livestock health. After winning a 2017 Inspire Challenge start up grant, the project created a digital community, the Africa Farmers Club, a Facebook group which now has over 135,000 members. For the first time, Kenyan farmers have access to information that can improve their farming practices via social networks.
Additionally, the Farm.ink team built a chatbot on the Facebook Messenger platform to deliver dairy farmers information on productivity, markets, and livestock management. A survey by the project found that 92 percent of dairy farmers reported changing their farming practices based on information received through this service.
Moving forward with funds from the 2018 Inspire Challenge scale up grant, the project will expand their digital services up and out across Africa. In partnership with IDEO.org, the team will build a web platform to enable farmers across multiple value chains, such as maize and tomato, to ask questions, share advice, and access expert agronomic content.
The project also seeks to build a machine-learning classifier that would turn the hundreds of thousands of Africa Farmers Club posts relating to livestock in Kenya into research insights about livestock health in the country. Farm.ink will combine these insights with International Livestock Research Institute (ILRI) data to deliver timely, targeted information to small-scale dairy farmers in East Africa through their mobile phones.
Step by step
The project was one of five winners of the Inspire Challenge 2017 and was awarded US$100K at the inaugural annual convention of the CGIAR Platform Big Data in Agriculture, 19-22 September 2017.
Building a natural language processing classifier
The team built a natural language processing (NLP) classifier to accurately label incoming data. Farm.ink completed multiple workshops with the ILRI team to create a data labeling protocol. Using this protocol, hundreds of thousands of posts, comments and images generated by Kenyan farmers were analysed and labelled over 26,000 rows of data using a human-in-the-loop labeling method. They then built and trained a NLP-based classifier to label livestock posts (e.g. health issue, buying/selling).
This work brought to light various problems and concerns of livestock farmers in Kenya. For instance, the data revealed that 40 percent of the information shared relates to the buying and selling of cattle. Drilling down one level and looking into the animal husbandry category, the most popular topic of discussion is feeds.
Developing analytical tools to turn the data into actionable information
The team built open-source analytical tools to turn the data into actionable information for dairy farmers and scientific researchers. Using the labelled data, the team created a dashboard tool to analyse data trends that the ILRI team and others can access. The tool, along with the labelled data, will be publicly accessible by the end of 2018, enabling others to build on the results.
Creating a dairy chatbot service
The team designed a simple and engaging way for farmers to receive information on their phones. Over the last year, the team has prototyped and tested multiple chatbot features for livestock farmers using the labelled data. They have combined actionable information from the social feed data with ILRI data to create targeted farmer alerts through the chatbot. These alerts include information on animal health issues, an analysis of local milk prices, and a report on local cows for sale.
Real results on the ground
The userbase of the Africa Farmers Club has reached 24,000 dairy farmers and is on track to exceed 40,000 by the end of 2018. A recent survey of 406 dairy chatbot users found that 92 percent reported having changed the way they farm based on information received through our services.
US$125K scale-up grant
The project was a runner up in the Inspire Challenge Scale Up 2018 and was awarded US$125K at the second annual convention of the CGIAR Platform for Big Data in Agriculture, 3-5 October 2018.
Understanding the key needs of digital farmers
Farm.ink worked in deep collaboration with partner IDEO.org to understand the key needs of digital farmers. In-depth interviews were conducted with 28 farmers, including those who had used Farm.ink services before and those who had not.
The key needs identified from conversations were:
- a better theoretical foundation for farming
- social safe space, as compared with current online forums
- more practical farming education
These user-generated themes were used to begin prototyping web platform concepts.
The team took a design-thinking approach to prototyping products that optimised for three factors:
- Desirability: Would users engage with the products initially and over time?
- Viability: What business model options are on the table? Can we simulate real payments?
- Feasibility: How can we move from lightweight prototypes to more functional ones that incorporate real data?
In collaboration with IDEO.org, the team reviewed the design research to develop 14 concepts, which were boiled down into four initial prototypes to test with farmers.
In early 2019, the team tested over ten prototypes, with over 30 user feedback sessions with farmers. Each hour-long session was recorded in detail, where possible, using user testing software Lookback.
The team rapidly prototyped and tested these ideas, generating new working prototypes for farmers to see in seven day cycles. The photos below show some of the many tested prototypes.
Farmer impact journey
As the prototyping phase matured, the team built up a design specification for the public beta. The team has also tested a range of technology options (e.g. native apps vs. websites).
This initial phase helped the team better understand the potential farmer impact journey for a user interacting with the new platform.
Now, the team will move into the build phase, turning the early prototypes into a public beta. The team aims to have this live with an early group of beta testers in June 2019.
Games and learning
Using a continuous customer development cycle (i.e. testing with users every month), the team continues to gain key learnings from users. With a focus on learning and social farming Q&A combined, we found that:
- Farmers ask better questions and respond with better answers when their initial experience of the platform is an educational one.
- Incorporating game mechanics hugely increased user time spent on the platform in unbroken periods of use.
- Front-loading the user experience with educational agricultural information provides incredible early impact data on what farmers know about key topics, and how quickly they learn (which is fast!).
User testing isn’t rocket science; the team looked out for when users gave a beaming smile while using the latest version.
Technology stack choices
The team decided the key pieces of the tech stack:
- Progressive web apps offer a range of functions such as push notifications and offline modes to web app technology.
- M-PESA allows the team to integrate a payment channel for farmers to easily send and receive funds on the platform.
Fall Armyworm course
In the scale-up phase, Farm.Ink’s aims to work with new content areas. Fall Armyworm is currently causing widespread damage to maize crops across Africa, so the pest has been a new area of focus for the team.
The team won the FAW Tech Prize challenge to build digital tools for farmers to manage this critical pest.
We translated Fall Armyworm content produced by Feed the Future and CIMMYT into a format that proved to be highly engaging with our users and led to stimulating conversation among users. These types of conversations create precisely the kind of social discussion data that can be analyzed to understand broader trends and insights.
The team have started prototyping an API that allows partners to rapidly deliver information to a farming audience. Key impact data about the uptake and learning outcomes among farmer users is tracked.
The team aims to make this a simple but effective solution for partners. It can have a positive and measurable impact on smartphone-owning farmers who are thirsty to learn!