From 2008-2012, I headed Grockit's research and development efforts, leading a research-based, experiment-driven approach to building the company’s core product, a web application supporting live collaborative learning.
- I identified promising techniques and models from the statistics research community and used these to develop algorithms for adaptive assessment and adaptive instruction.
- I worked with Eric Ries in 2009-2010 to develop an infrastructure to rapidly introduce and effectively communicate results from randomized controlled experiments (aka split tests.) Discussed in his chapter in The Lean Startup about Grockit.
- I built the company's first platform for learning analytics.
- I authored a number of peer-reviewed publications.
- I co-organized conference workshops on intelligent and adaptive behavior in collaborative learning systems with researchers from CMU and ASU in 20101 and 20112.
- I collaborated with school districts to design pilot and efficacy studies.3
- I hired talented people, including one data scientist and two doctoral students for summer research internships. (Internships focused on improvements to our predictive model, which fared well as the benchmark in the Kaggle data science competition that we hosted.)
- I created Grockit Answers, a tool for hosting Question & Answer discussions around educational web videos. Grockit Answers demonstrated a new way to support peer collaboration: pseudo-synchronous learning.
My Grockit Publications
Self-directed learners value the ability to make decisions about their own learning experiences. Educational systems can accommodate these learners by providing a variety of different activities and study contexts among which learners may choose. When creating a software-based environment for these learners, system architects incorporate activities designed to be both effective and engaging. Once these activities are made available to students, researchers can evaluate these activities by analyzing observed usage and performance data by asking: Which of these activities are most engaging? Which are most effective? Answers to these questions enable a system designer to highlight and encourage those activities that are both effective and popular, to refine those that are either effective or popular, and to reconsider or remove those that are neither effective nor popular. In this paper, we discuss Grockit – a web-based environment offering self-directed learners a wide variety of activities – and use a mixed-effects logistic regression model to model the effectiveness of nine of these supplemental interventions on skill-grained learning.
Web-based learning systems offer researchers the ability to collect and analyze fine-grained educational data on the performance and activity of students, as a basis for better understanding and support- ing learning among those students. The availability of this data enables stakeholders to pose a variety of interesting questions, often specifically focused on some subset of students. As a system matures, the number of stakeholders, the number of interesting questions, and the number of relevant sub-populations of students also grow, adding complexity to the data analysis task. In this work, we describe an internal analytics system designed and developed to address this challenge, adding flexibility and scalability. Here we present several examples of typical examples of analysis, discuss a few uncommon but powerful use-cases, and share lessons learned from the first two years of iteratively developing the platform.
Grockit provides a place for students to master new concepts and exercise what they learn through a set of study modes designed to accommodate a variety of learning styles and learner preferences. These include: (1.) small group study, which leverages the power of collaborative learning dynamics to provide students with a social learning network that can help motivate and assist them, (2.) individual study, which builds and uses a data-driven model of a student's abilities to provide that student with appropriate challenges for learning, (3.) instructor-led classes, which draw on a teacher's domain knowledge and experience to provide a guided and structured path for groups of learners.
Supplementary Proceedings of the Tenth International Conference on Intelligent Tutoring Systems (ITS-2010), Workshop on Opportunities for intelligent and adaptive behavior in collaborative learning systems.
One unique characteristic of learning systems that support peer collaboration is that these systems have the potential to supplement or replace software-based representations of domain- and learner-models with the representations implicitly formed by peers. In order to realize this potential, a collaborative activity must sufficiently motivate peers to reflect, collect, and communicate these mental models. Peer-assessment represents a class of activities that address this challenge by design. In this work, we describe a project, currently under development, in which peer-assessment is melded with peer-instruction to create a new learning activity for an existing collaborative learning platform. We present the rationale behind the design of the activity, focusing specifically on how it draws from and synthesizes the three modes of learning supported by the Grockit platform: adaptive individual study, live collaborative small-group study, and instructor-led skill-focused lessons. By treating teaching as a demonstration of learning, we illustrate how a single activity can peer-assess mastery and peer-assist learning.
Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2009 (E-LEARN 2009).
While many web-based learning systems connect students asynchronously, fewer systems focus on facilitating synchronous interactions among learners. Given the value of real-time communication – the social and motivational benefits of having a cohort of peers and the ability for a student to get immediate answers to pressing questions – it is perhaps surprising that more systems do not support interaction synchronicity. We suggest that this is due, in part, to a mismatch between the hypertext document-oriented nature of the web and the social activity-oriented nature of learning, and we explore how several systems address this discrepancy. We discuss Grockit, a web-based learning environment that we designed to support both synchronous and asynchronous interactions, and share lessons learned from grappling with the choices enabled by this flexibility: Which interactions should to be synchronous? Which should be asynchronous? Which should be a mix? What should that mix be?
Supplementary Proceedings of the 14th International Conference on Artificial Intelligence in Education (AIED-2009). Intelligent Educational Games workshop, 2009.
The recent movement towards publishing open educational resources has increased the variety and quantity of learning materials available to students outside of the traditional classroom environment. Several core characteristics of the classroom environment, however, are difficult to offer through a web-based interface, including: (1) interaction and camaraderie among a cohort of peers, (2) the ability to get "real-time" answers to pressing questions, and (3) a motivating force to keep the student engaged over time. An online learning environment can approximate the value of peer cohorts and live question-answering by supporting (and encouraging) synchronous interactions among individuals studying a common topic. A learning system can motivate participation and collaboration by incorporating elements of game mechanics in the activity. We discuss Grockit, a recently-launched website that combines a virtual study group format with multi-player game dynamics to provide an engaging live collaborative learning environment for geographically-dispersed learners.
I co-organized a workshop with researchers at Carnegie Mellon University at the 10th International Conference on Intelligent Tutoring Systems (ITS-2010) on Opportunities for adaptive behavior in collaborative learning systems. ↩
Building on the success of the 2010 workshop, I collaborated with researchers at Carnegie Mellon University and Arizona State University in organizing a workshop at the Fifteenth International Conference on Artificial Intelligence in Education (AIED-2011) on Opportunities for Intelligent Behavior in Collaborative Learning Systems. ↩
We were named as a Finalist in the Gates Foundation-funded Next Generation Learning Challenges for a project designed to demonstrate and measure the impact of Grockit's collaborative learning platform for students pursuing self-paced coursework in a virtual school setting. ↩