Pushing the Limits of Artificial Intelligence and Natural Language: How Creating an Incubation Team has Enabled Narrative Science to Stay at the Forefront of Innovation
By: Nate Nichols
One of the best parts of my job at Narrative Science (NS) is leading our Incubator program. Like other successful software companies, NS has a constant focus on making our products more valuable to users. A major component of this is maintaining a roadmap of potential features and improvements, informed by research into users and customers. A laser-like focus on the most valuable thing we can provide next is critical but also introduces the risk of missing the forest for the trees. Ideas that are experimental, likely to fail, or undefined can be difficult to accommodate in a model focused on continually adding incremental value. Of course, it’s exactly these same ideas that are also often the most impactful. Besides regular companywide hackathons, we’ve created an internal Incubator program specifically to explore these types of ideas that are too risky or undefined to slot into a roadmap.
About the Program
It’s easy to say “We should have an R&D program”, but it’s much harder to actually pull the trigger and get one started. It’s tempting to consider such a program a “nice to have” – something your org will take on in the vague future when you suddenly find yourself with more money and employees than you need. Not surprisingly, this happy future can feel just a quarter or two away for years. We decided in mid-2017 to put our money where our mouth was and form our Incubator program, based primarily on two beliefs:
- Our product, Lexio, was sufficiently far along for us to be able to move high-performers off of Lexio and into Incubator.
- Our core IP is a strategic asset of the company and should be granted a commensurate commitment.
Since its founding two years ago, our program has heavily influenced our thinking, our roadmap, and our strategic vision for the company and the types of value we can ultimately provide our users. Our Incubator team members are also inventors on more than a dozen patents coming out of this work. Not bad for a team of two to three people! I’ve thought about what has made our Incubator function successful, and I believe there are ultimately three things required.
3 Key Ingredients for Success
1) Cross-company focus on innovation
The first is that everyone, from the C-suite down, needs to be aligned with the goals of the Incubator function and have a shared view of what success looks like. The objective for our Incubator program is to push the limits of our products and tackle complex and revolutionary projects within artificial intelligence, natural language, and more. This helps our products stay at the forefront of innovation while still building near-term needs for our user base.
Of course, keeping a dedicated group of people working on ideas that might not work requires a lot of trust from senior management. I’ve found the best way to maintain this trust is by tying everything we do back to value to our users. We don’t take on projects because they’re neat or fun to work on, we take them on because we believe they may provide huge additional value, despite not knowing enough about the idea or approach to slot it into our ongoing roadmap.
2) A process that allows us to take risks and move fast
The second critical component is having the right operational model. For us, that’s two-month-long projects, built by a stable team of Incubator personnel. For each project, we also include an additional domain expert. This is typically an NS employee who has expertise in areas that are particularly relevant to that specific project (e.g. machine learning or data science). Besides bringing a lot of focused expertise and experience to the project, it also gives the domain expert a break from their typical work and lets them have a lot of impact on a more exploratory project.
Our operational model also includes relaxing our usually strict engineering requirements. Our Incubator projects focus on the risky or unknown parts of the problem we’re tackling, so concerns around coding best practices, browser compatibility, usability, maintainability, scalability, etc. are set aside. We know how to do those already, and it’s critical that the Incubator spends as much of our two-month-long development cycles pushing on the parts of the problem that we don’t know.
3) The right team
The third critical component is having the right people on the team. Being successful on the Incubator team requires a willingness and ability to move quickly, work with a large amount of uncertainty, and avoid getting bogged down in minutiae. Not everyone has these types of skills, and not everyone is interested in this kind of environment. The key is to find the people who do have these skills and who are interested in this environment, point them at big and poorly understood problems, and then stay out of their way.
Meet the team:
As a Distinguished Principal – Product Strategy and Architecture, Nate is responsible for high-level design of how our products work, particularly around what our products know, how they learn more, and how they apply what they know to provide value to our users. To ensure Narrative Science delivers the most value to our customers using the most advanced technology available, he works closely with industry leaders and academia to incorporate new techniques and approaches, as well as articulating our vision for how AI can help people better understand the world around them. He also oversees our Incubator program, a cross-functional team dedicated to exploring innovative technology that will propel our customers forward.
Mike Smathers is a Principal Engineer at Narrative Science and the engineering lead of the Incubator program. He’s responsible for conceptual/architectural/UX design, rapid prototyping, and mentorship of junior engineers. His 8+ years at the company have been spent developing best practices in system architecture, API design, and front-end engineering.
Daniel Platt is a Senior Principal at Narrative Science. In addition to serving as a Senior Product Manager on Narrative Science’s Lexio Product, he is the lead PM of the company’s Incubator Group. Dan created the first NLG extensions into business intelligence applications in 2015. Dan was one of the first employees of Narrative Science, joining in 2010 after receiving a Masters in Journalism from Northwestern University, and has served in a variety of roles including professional services and customer training and education.
It’s easy to think of an Incubator program as a “nice to have” or something your company can afford to support at some point in the future. In some industries or at some companies that may be the case,but at Narrative Science, we’re working to fundamentally transform the way people understand and communicate the data that’s important to them. With a goal that big, we can’t afford to focus 100 percent on the next piece of incremental value we can provide. To truly serve our users, we need to keep one eye on the horizon and always be working to understand the value, capabilities, and experiences we should be providing in the future.
I am incredibly excited to share that we are starting to see our Incubation projects begin to materialize in our products. Our newest product, Lexio, will have one of our Incubator projects in production within the next three months. Stay tuned for our next blog post, where we will give you a sneak peek of our new project and why we are so excited for what’s to come.