loading
loading
The ChinaTalk team is going to CoRL (the Conference on Robot Learning) in Austin! Reply to this email if you’d like to hang out between November 9th and 12th. We’re also doing karaoke on the 11th — RSVP here if you’d like to join! A. Chen writes about people, technologies, and their relationships to power. Her submission, “Take Ten-Thousand: Egocentric Data Collection for Humanoid Robots,” won one of 7 Grand Prizes at the ChinaTalk Essay Contest. We’re thrilled to share this report from the frontlines of China’s robot data collection industry! Where is the center of the universe? For the residents of Beijing, the answer is Wudaokou. The neighborhood houses China’s top universities, research institutions, and labs, drawing talent and funding from every corner of the country. The streets are radiant at night, packed full of bars and clubs that have somehow bounced back after the COVID-19 pandemic. And the number 13 metro line, which rattles overhead every five minutes, is never empty. I came to Wudaokou to understand China’s AI industry — particularly, the hype surrounding humanoid robots, and the methods used to train them. Everyone has seen videos of these robots: eerily tall and lanky, some of them shedding nuts and bolts as they sprint down a track, others backflipping repeatedly on a dance stage. The global market for humanoid robotics, or embodied intelligence, is said to be worth $38 billion by 2035: the latest chapter in the U.S.-China technology race that has dominated international headlines for over a decade. This year, China’s 15th Five-Year Plan declared embodied intelligence a matter of national priority, then laid a strategy for developing training grounds across the country. Over 150 humanoid robotics companies and 40 state-owned robot data collection centers were reported to be in operation in China by the end of 2025. According to the Global Times, Chinese embodied intelligence startups drew 50 billion yuan in investment deals by September 2025: an increase of 2.5 times from the previous year. More recently, embodied intelligence startups have begun to focus on the human hand, whose articulation and coordination has yet to be replicated by robots. Data of the human hand can’t be generated passively, in the same way that an Instagram user’s scroll or like might be harvested, and researchers of embodied intelligence are racing to solve this dilemma with a number of techniques. Most recently, videos have begun to circulate of people wearing headsets and filming themselves at work — a technique called egocentric data collection — to train robot hands. I sat down with the researchers and entrepreneurs behind China’s egocentric data collection space to understand the industry. Beneath the speculation, the social media virality, and the staggering amounts of investor money, what — and who — does it actually take to train a robot? The Third Wave of Data Collection Dr. Jackson Guo, a researcher at the Beijing-based Kunlun Nest Robot Open Laboratory (昆仑巢人机器人开放实验室), invited me to his workspace to discuss the history and technicalities of egocentric data collection. He identifies three distinct periods of robot data collection: teleoperated, isomorphic, and egocentric. Teleoperated data collection, the first generation of data collection techniques, appeared around 2017. That year, Stanford Professor Fei-Fei Li piloted a program that allowed users to connect their phones to remote controls that could control robot arms. Teleoperated data collection methods have since become more streamlined: a user stands several feet away from a robot and moves its body using remote controls, sometimes with a VR headset to simulate a different environment. According to Dr. Guo, the individuals responsible for teleoperated data collection often suffered from fatigue and joint pain after hours of performing slow, repetitive motions. Teleoperation was also expensive; hardware could cost hundreds of thousands of yuan per set. Even so, companies such as Helix, OpenXE, and PI (Physical Intelligence) collected tens of thousands of hours of data using this method. Most of the data collected was for gripper hands, which operated on a simple binary command: 0 for closed, 1 for open. Fine-grained, dexterous manipulations — such as cutting an apple, folding a shirt, or playing the piano — proved too complex for a user to simulate through a remote control. In 2024, Stanford researchers Cheng Chi and Zhenjia Xu and Professor Shuran Song published a thesis on Universal Manipulation Interface (UMI). This marked the second generation of data collection techniques, or isomorphic data collection, which took Silicon Valley by storm. Using Velcro straps, elastics, or metal hinges, human subjects connected their hands to the inside or underside of a robotic exoskeleton, feeding information to a robot more swiftly and precisely than a teleoperated remote control could. Gripper hands were still the easiest model to train, as exoskeletons were often too rigid or heavy to allow for more articulated movements. Dr. Guo credits Generalist AI with the most successful isomorphic data collection program in the U.S.: within six months, they used one-thousand workers to collect over 250,000 hours of data. By the start of 2026, egocentric data collection had begun to define a third generation of data collection techniques. On February 18, 2026, NVIDIA introduced the concept of egocentric data collection in its EgoScale Project, which shifted the primary method of data collection from isomorphic exoskeletons to camera footage. NVIDIA’s team attached cameras to the foreheads and wrists of over 1,000 people, which they used to film 20,854 hours of first-person footage: mostly everyday tasks, such as washing dishes and cutting vegetables. Researchers mapped the positions of the subjects’ wrists, fingers and joints through each frame of the footage. This preliminary data was then augmented and refined using a variety of isomorphic techniques. In their paper, the NVIDIA team claimed that training a robot on egocentric data increased the chances of it successfully folding a shirt by 88%. “Humans can be treated as a scalable embodiment for learning general embodied intelligence,” celebrated the authors of the paper, before encouraging future researchers to record as much video as possible for robot training. Imperfect and low-quality footage can always be cleaned: what matters is getting enough video to clean. Today, among most companies in the egocentric data business, the consensus is, the more, the merrier. As was the case for teleoperated and isomorphic data collection, egocentric techniques quickly latched on across the U.S., before spilling across the world to China. The U.S.-based EgoVerse uses Meta’s Project Aria glasses, or their Mecka app downloaded to an iPhone 15+, to record an individual in a variety of settings. Build AI has raised over $30 million with the goal of collecting over 1 billion hours of egocentric footage by 2028, a goal shared by a dozen other startups in Silicon Valley. And in an interview with Wall Breaker, World Intelligence founder Grace Zhang declared her ambitions of creating the world’s biggest physical AI data marketplace. “I want people to film and record data of themselves doing everyday chores in their homes,” she said. “That is my ideal future.” The Chinese companies jumping on the trend Today, Chinese companies have been collecting egocentric data for their own in-house use, or to sell to domestic and overseas consumers. Jingdong, or JD.com, was one of the first Chinese companies to jump on the egocentric data collection wave. On March 16, 2026, founder and chairman Liu Qiangdong announced the creation of a data collection center in his hometown of Suqian (宿迁), Jiangsu Province, with the goal of collecting over 5 million hours of footage by 2027 and 10 million hours of footage by 2028. Over 100,000 Suqian citizens have signed up to film themselves performing five categories of everyday tasks — logistics and warehousing, industrial manufacturing, healthcare, home services, and urban operations — in 100 distinct environments and scenarios. They use JD.com’s own headgear, which is said to weigh less than a pair of glasses. But Liu’s ambitions don’t stop there. In August 2026, JD.com launched their JoyAI video editing program, which can use preexisting egocentric footage to generate even more videos capable of training robots. The program can alter the background or lighting of an original video, simulate environments where it may be too dangerous or inconvenient to bring a camera, or turn human hands into robot hands onscreen. Perhaps the egocentric data collection industry is already starting to cannibalize itself. Or, perhaps, this is a vital new direction for an industry still grappling with high production costs and a fluctuating market. Navigating the data market Because, despite all the hype and speculation, egocentric data collection is still an unpredictable industry. An anonymous industry expert claims that both collectors and buyers are hesitant about the industry’s scalability and sustainability, due to the cost of creating usable data. “People wouldn’t hesitate to buy egocentric data if it were the cost of water (5 to 10 yuan per unit),” he tells me, “but I don’t think the cost of collecting data can be lowered any further than what it is today.” He places the cost of collecting one hour of raw data at around 100 yuan, and the cost of generating usable data at around 500-700 yuan per hour. Dr. Iris Wang, an entrepreneur and researcher at Tsinghua University, sat down with me to discuss the business side of the industry. “The market decides the direction of the industry and innovation,” she says, “not the other way around.” In the vast ecosystem of data collection — where data is sold to startups and tech companies, and where these startups and tech companies sell finished robots to consumers — the vendors of egocentric data lack truly substantial bargaining power. The needs of the consumer still dictate the direction of data collection and research. For instance, while academic research labs might want to purchase dexterous hands trained on egocentric data, most warehouses and commercial companies are still happy to use binary gripper hands trained on isomorphic data. “The second generation of data collection — isomorphic exoskeletons and UMI — is still very popular among Chinese buyers,” says Dr. Guo. “Unless egocentric data collectors can harvest, say, 10 billion hours of data, we can’t outperform UMI in the market.” The anonymous industry expert agrees, estimating that about 30% of the robot market still only seeks to purchase second-generation UMI-based data for gripper hands. “I wish the market would give us a clearer direction,” he tells me. “That, more than anything, will decide the future of data collection.” Egocentric data is not a versatile product. Footage is difficult, if not impossible, to resell: customers often have specific requirements for the lighting, background, and movements for every video, and customized data is often protected by exclusive IP. In response, many companies collect data solely for in-house use, creating complex and fluctuating dynamics across the supply chain. But Chinese companies might find other opportunities in the global supply chain of data. Chinese data aggregators often serve as the middleman between Global South countries and their Global North clientele, selling footage collected from all over Asia (including China itself) to the U.S. and Europe. “Labor costs are quite low in Asia,” says Edwin Yap, the founder and CEO of Mr. Robot, a data collection and robotics company based in six countries across Southeast Asia. Isomorphic data collection is still too expensive to be the basis of a sustainable business model, so Mr. Yap uses his egocentric footage to train his own robots, and to sell to data aggregators in other countries. All my interviews frequently returned to the cost of production and labor. On paper, egocentric data collection seems cheaper than both isomorphic and teleoperated methods: after all, there’s no cost of production needed for a giant exoskeleton or remote controls. But the cost of training workers, particularly at the scale needed to reach 10 billion hours of footage, is still considerable. While major companies such as JD.com, ByteDance, Ant Group, and Xiaomi have the financial resources to conduct extensive pre-training, collect usable footage, and purchase the necessary data, startups in the industry face a much more uncertain path. According to Dr. Wang, startups raising over 1 billion RMB will usually pursue egocentric data used for pre-training, while those raising less than 1 billion RMB focus on post-training methods that require little to no ego-centric data. Our talk — and the news of JD.com’s data collection center — drew me back to the question of labor: whose data finds its way into algorithms, and what part of the reward do they reap? The hands behind the camera A quick scroll on Xiaohongshu, one of China’s foremost social media platforms, reveals hundreds of advertisements posted within the last month: recruiters offering anywhere from 100 to 200 yuan per day for workers willing to travel to a data collection center and film themselves using egocentric hardware and techniques. I did not find any adverts affiliated with big internet companies or startups: rather, these posts were published by small actors looking to sell data to more powerful buyers. Some adverts ask for workers to arrive with experience in egocentric data collection; others offer training on the spot. More specific requirements — the workers’ age, eyesight, mobility, susceptibility to VR-induced vertigo — vary by recruiter. All the adverts boast reasonable working hours (9 AM to 6 PM, with a lunch break) and overtime pay. But none of them offer the permanent contracts or benefits that are associated with full-time work opportunities. Dozens of users debate the merits of the job in the adverts’ comments sections. “Seems boring,” remarks one commenter. “Seems sketchy,” quips another. Others are more curious, asking for specifics about the work and pay. One Xiaohongshu user offers a detailed day-in-the-life of a data collection subject: a 6:15 AM commute to a data collection center, a physical check-up for blood pressure and body weight, and an entire day spent copying hand movements off a tutorial video. The user needed to earn a complete quota of “data points” — measured by the accuracy of her movements compared to the tutorial video — to be paid in full. After a day and 40 minutes of overtime spent trying to hit this quota, she received 180 yuan ($27). “Most people just wanted to get their money and leave,” the user writes. “No one knew each other, and no one talked.” I was curious about the possibility of a shared class consciousness between data collection subjects, particularly in large and well-funded data centers such as JD.com’s, but came away with no conclusions: the industry, it seems, is still too new and fragmented. Egocentric data collection, especially when integrated into existing workplaces, raises the question of labor and privacy protections. New technologies often explode in scale before laws can catch up, and the selling of personal data has always been mired in a legal gray area. This is especially complicated in an international data market, where labor and privacy laws differ by country. Human Archive, a Redwood City-based startup, has partnered with a number of companies in India and Southeast Asia to distribute headsets to workers in home services, hotel, and restaurant sectors. They aim to sell this footage to AI labs and robot companies in the U.S. and Europe: the latest iteration of a worldwide supply chain that depends on cheap labor in the Global South. These workers receive about $1 an hour for participating in data collection. Human Archive claims to follow the law of whatever country they are collecting from, such as the Digital Personal Data Protection Act in India. But not all workplace-integrated data collection is so transparent. In April, a Scroll.In story revealed that Indian workers were not told that the egocentric videos they filmed would be used to train AI models. And in the same month, Puneet Jindal, the CEO of Labellerr AI, told CNN that many factory workers do not receive any information about how their data will be used when they start to film themselves. Following pressure from Bengaluru residents, India’s Ministry of Electronics and Information has been looking into the controversy surrounding home services startup Pronto, which announced that it had launched an opt-in recording feature for workers performing household tasks. It is unclear whether Pronto is selling the data collected from these recordings to buyers. In China, workplace-integrated data collection is still rare. Chinese data collectors recruit subjects to film themselves at specialized centers, or from their own homes, instead of attaching cameras to workers in their workplaces. The Chinese model, as I’ve gleaned from interviews and Xiaohongshu adverts, is to generate high-quality footage that doesn’t need much cleaning or processing. This stands in stark contrast to the low-quality footage collected in workplaces in South and Southeast Asia: data which may be cheaper to collect, but more expensive to clean. From Build AI’s website, which claims to be “hypergambling” with a “small and exceptionally cracked team” to “unlock human potential” — to Human Archive’s, which claims to erase all “undesirable physical work” and to usher in “a new era of abundant creativity” at this “once-in-a-generation inflection point” — American rhetoric around egocentric data collection is big, bragging, and triumphant. This is a far cry from the tentativeness I witnessed in Chinese founders and researchers, the majority of whom are still waiting for the market to decide the next direction — or generation — of data collection methods. This is even farther from the sentiments of the humans whose footage become the data, which range from apathy to suspicion to resistance. At the end of our talk, Dr. Guo turned to me and grinned. “The U.S. crosses the river by feeling the stones, and China crosses the river by feeling America,” (美国摸着石头过河，中国摸着美国过河) he said, a quip that made me laugh out loud. It’s a play on Chen Yu and Deng Xiaoping’s famous idiom: improvising amidst the rapids, seeking stability wherever it may be found. In Dr. Guo’s eyes, it is the U.S. that is feeling for the stones and China that is feeling for the U.S., looking for stable ground in the shadow of American startups and labs. The narrative is not as simple as U.S.-China competition. There is a heavy-handed metaphor to be made about the data that crosses not only a river, but an ocean, from India to China to Silicon Valley. About the rivers of power and money that reach far beyond any individual’s comprehension. My interviews clarified an industry that is often mysticized by hype and technical lingo. They also exposed me to a sprawling supply chain that is both extremely unpredictable and historically embedded in our current world order. I left Wudaokou with two questions: What are we overlooking in our rush to cross the river? And does the same shore await us all on the other side?
The ChinaTalk team is going to CoRL (the Conference on Robot Learning) in Austin! Reply to this email if you’d like to hang out between November 9th and 12th. We’re also doing karaoke on the 11th — RSVP here if you’d like to join! A. Chen writes about people, technologies, and their relationships to power. Her submission, “Take Ten-Thousand: Egocentric Data Collection for Humanoid Robots,” won one of 7 Grand Prizes at the ChinaTalk Essay Contest. We’re thrilled to share this report from the frontlines of China’s robot data collection industry! Where is the center of the universe? For the residents of Beijing, the answer is Wudaokou. The neighborhood houses China’s top universities, research institutions, and labs, drawing talent and funding from every corner of the country. The streets are radiant at night, packed full of bars and clubs that have somehow bounced back after the COVID-19 pandemic. And the number 13 metro line, which rattles overhead every five minutes, is never empty. I came to Wudaokou to understand China’s AI industry — particularly, the hype surrounding humanoid robots, and the methods used to train them. Everyone has seen videos of these robots: eerily tall and lanky, some of them shedding nuts and bolts as they sprint down a track, others backflipping repeatedly on a dance stage. The global market for humanoid robotics, or embodied intelligence, is said to be worth $38 billion by 2035: the latest chapter in the U.S.-China technology race that has dominated international headlines for over a decade. This year, China’s 15th Five-Year Plan declared embodied intelligence a matter of national priority, then laid a strategy for developing training grounds across the country. Over 150 humanoid robotics companies and 40 state-owned robot data collection centers were reported to be in operation in China by the end of 2025. According to the Global Times, Chinese embodied intelligence startups drew 50 billion yuan in investment deals by September 2025: an increase of 2.5 times from the previous year. More recently, embodied intelligence startups have begun to focus on the human hand, whose articulation and coordination has yet to be replicated by robots. Data of the human hand can’t be generated passively, in the same way that an Instagram user’s scroll or like might be harvested, and researchers of embodied intelligence are racing to solve this dilemma with a number of techniques. Most recently, videos have begun to circulate of people wearing headsets and filming themselves at work — a technique called egocentric data collection — to train robot hands. I sat down with the researchers and entrepreneurs behind China’s egocentric data collection space to understand the industry. Beneath the speculation, the social media virality, and the staggering amounts of investor money, what — and who — does it actually take to train a robot? The Third Wave of Data Collection Dr. Jackson Guo, a researcher at the Beijing-based Kunlun Nest Robot Open Laboratory (昆仑巢人机器人开放实验室), invited me to his workspace to discuss the history and technicalities of egocentric data collection. He identifies three distinct periods of robot data collection: teleoperated, isomorphic, and egocentric. Teleoperated data collection, the first generation of data collection techniques, appeared around 2017. That year, Stanford Professor Fei-Fei Li piloted a program that allowed users to connect their phones to remote controls that could control robot arms. Teleoperated data collection methods have since become more streamlined: a user stands several feet away from a robot and moves its body using remote controls, sometimes with a VR headset to simulate a different environment. According to Dr. Guo, the individuals responsible for teleoperated data collection often suffered from fatigue and joint pain after hours of performing slow, repetitive motions. Teleoperation was also expensive; hardware could cost hundreds of thousands of yuan per set. Even so, companies such as Helix, OpenXE, and PI (Physical Intelligence) collected tens of thousands of hours of data using this method. Most of the data collected was for gripper hands, which operated on a simple binary command: 0 for closed, 1 for open. Fine-grained, dexterous manipulations — such as cutting an apple, folding a shirt, or playing the piano — proved too complex for a user to simulate through a remote control. In 2024, Stanford researchers Cheng Chi and Zhenjia Xu and Professor Shuran Song published a thesis on Universal Manipulation Interface (UMI). This marked the second generation of data collection techniques, or isomorphic data collection, which took Silicon Valley by storm. Using Velcro straps, elastics, or metal hinges, human subjects connected their hands to the inside or underside of a robotic exoskeleton, feeding information to a robot more swiftly and precisely than a teleoperated remote control could. Gripper hands were still the easiest model to train, as exoskeletons were often too rigid or heavy to allow for more articulated movements. Dr. Guo credits Generalist AI with the most successful isomorphic data collection program in the U.S.: within six months, they used one-thousand workers to collect over 250,000 hours of data. By the start of 2026, egocentric data collection had begun to define a third generation of data collection techniques. On February 18, 2026, NVIDIA introduced the concept of egocentric data collection in its EgoScale Project, which shifted the primary method of data collection from isomorphic exoskeletons to camera footage. NVIDIA’s team attached cameras to the foreheads and wrists of over 1,000 people, which they used to film 20,854 hours of first-person footage: mostly everyday tasks, such as washing dishes and cutting vegetables. Researchers mapped the positions of the subjects’ wrists, fingers and joints through each frame of the footage. This preliminary data was then augmented and refined using a variety of isomorphic techniques. In their paper, the NVIDIA team claimed that training a robot on egocentric data increased the chances of it successfully folding a shirt by 88%. “Humans can be treated as a scalable embodiment for learning general embodied intelligence,” celebrated the authors of the paper, before encouraging future researchers to record as much video as possible for robot training. Imperfect and low-quality footage can always be cleaned: what matters is getting enough video to clean. Today, among most companies in the egocentric data business, the consensus is, the more, the merrier. As was the case for teleoperated and isomorphic data collection, egocentric techniques quickly latched on across the U.S., before spilling across the world to China. The U.S.-based EgoVerse uses Meta’s Project Aria glasses, or their Mecka app downloaded to an iPhone 15+, to record an individual in a variety of settings. Build AI has raised over $30 million with the goal of collecting over 1 billion hours of egocentric footage by 2028, a goal shared by a dozen other startups in Silicon Valley. And in an interview with Wall Breaker, World Intelligence founder Grace Zhang declared her ambitions of creating the world’s biggest physical AI data marketplace. “I want people to film and record data of themselves doing everyday chores in their homes,” she said. “That is my ideal future.” The Chinese companies jumping on the trend Today, Chinese companies have been collecting egocentric data for their own in-house use, or to sell to domestic and overseas consumers. Jingdong, or JD.com, was one of the first Chinese companies to jump on the egocentric data collection wave. On March 16, 2026, founder and chairman Liu Qiangdong announced the creation of a data collection center in his hometown of Suqian (宿迁), Jiangsu Province, with the goal of collecting over 5 million hours of footage by 2027 and 10 million hours of footage by 2028. Over 100,000 Suqian citizens have signed up to film themselves performing five categories of everyday tasks — logistics and warehousing, industrial manufacturing, healthcare, home services, and urban operations — in 100 distinct environments and scenarios. They use JD.com’s own headgear, which is said to weigh less than a pair of glasses. But Liu’s ambitions don’t stop there. In August 2026, JD.com launched their JoyAI video editing program, which can use preexisting egocentric footage to generate even more videos capable of training robots. The program can alter the background or lighting of an original video, simulate environments where it may be too dangerous or inconvenient to bring a camera, or turn human hands into robot hands onscreen. Perhaps the egocentric data collection industry is already starting to cannibalize itself. Or, perhaps, this is a vital new direction for an industry still grappling with high production costs and a fluctuating market. Navigating the data market Because, despite all the hype and speculation, egocentric data collection is still an unpredictable industry. An anonymous industry expert claims that both collectors and buyers are hesitant about the industry’s scalability and sustainability, due to the cost of creating usable data. “People wouldn’t hesitate to buy egocentric data if it were the cost of water (5 to 10 yuan per unit),” he tells me, “but I don’t think the cost of collecting data can be lowered any further than what it is today.” He places the cost of collecting one hour of raw data at around 100 yuan, and the cost of generating usable data at around 500-700 yuan per hour. Dr. Iris Wang, an entrepreneur and researcher at Tsinghua University, sat down with me to discuss the business side of the industry. “The market decides the direction of the industry and innovation,” she says, “not the other way around.” In the vast ecosystem of data collection — where data is sold to startups and tech companies, and where these startups and tech companies sell finished robots to consumers — the vendors of egocentric data lack truly substantial bargaining power. The needs of the consumer still dictate the direction of data collection and research. For instance, while academic research labs might want to purchase dexterous hands trained on egocentric data, most warehouses and commercial companies are still happy to use binary gripper hands trained on isomorphic data. “The second generation of data collection — isomorphic exoskeletons and UMI — is still very popular among Chinese buyers,” says Dr. Guo. “Unless egocentric data collectors can harvest, say, 10 billion hours of data, we can’t outperform UMI in the market.” The anonymous industry expert agrees, estimating that about 30% of the robot market still only seeks to purchase second-generation UMI-based data for gripper hands. “I wish the market would give us a clearer direction,” he tells me. “That, more than anything, will decide the future of data collection.” Egocentric data is not a versatile product. Footage is difficult, if not impossible, to resell: customers often have specific requirements for the lighting, background, and movements for every video, and customized data is often protected by exclusive IP. In response, many companies collect data solely for in-house use, creating complex and fluctuating dynamics across the supply chain. But Chinese companies might find other opportunities in the global supply chain of data. Chinese data aggregators often serve as the middleman between Global South countries and their Global North clientele, selling footage collected from all over Asia (including China itself) to the U.S. and Europe. “Labor costs are quite low in Asia,” says Edwin Yap, the founder and CEO of Mr. Robot, a data collection and robotics company based in six countries across Southeast Asia. Isomorphic data collection is still too expensive to be the basis of a sustainable business model, so Mr. Yap uses his egocentric footage to train his own robots, and to sell to data aggregators in other countries.
document fetched 2026-10-24 11:22 · canonical https://chinatalk.media/p/take-ten-thousand-egocentric-data