Research Scientist Intern, XR Input (PhD)
Meta

At Reality Labs Research, our goal is to explore, innovate and design novel interfaces and hardware for the next generation of virtual, augmented, and mixed reality experiences. We are driving research towards a vision of natural, seamless experiences in XR environments that are effective, enjoyable, and functionally indistinguishable from those in the real world. As a Research Scientist Intern, you will work with world-leading scientists spanning hardware, software, controls, human perception and action. You will research how augmented reality and wearables can be applied towards enterprise applications by teleoperating robots that can act from a distance. Your primary focus will be on connecting human hand actions in the virtual world (through wearables and mixed reality) to the actions of a robot hand. You will develop and leverage novel, robust, and sensor processing algorithms for wearables, using techniques such as self-supervised learning, robust decoding of biometric signals, confound mitigations, and sensor-fusion techniques. You will connect these algorithms to haptic feedback to give the human agent a connection to what the robot is doing. You will use human actions to control the robot and collect data from that interaction. Your work will be at the intersection of robotics, haptics, XR, signal processing, machine learning, and human-computer interaction, and you will have the opportunity to work with world-leading collaborators and mentors in these fields.Our internships are twelve (12) to twenty-four (24) weeks long and we have various start dates throughout the year. Some projects may require a minimum of 16 consecutive weeks.

Research Scientist Intern, XR Input (PhD) Responsibilities:

• Plan and execute cutting-edge research and design to achieve deliverables.
• Break down the generic problem statement to specific research areas that advance the state of the art.
• Develop an end-to-end demonstration experience with specific use cases.
• Modularize the experience to plug and play various control schema.
• Configure and commission off the shelf collaborative robots and connect them to XR headsets and research-grade wearables.
• Design UIs for mixing and matching feedback mechanisms.
• Gather user feedback on the efficacy of both the overall concept and specific multimodal techniques to provide interventions.
• Perform fundamental and applied research to push the scientific and technological frontiers of embodied artificial intelligence.
• Invent and/or improve novel data driven paradigms for embodied intelligence.
• Explore and conceptualize ways to leverage various data modalities (images, video, text, tactile, etc) and the roles they play in various levels of embodied reasoning and decision making.
• Investigate and develop new paradigms for delivering a spectrum of embodied behaviors, including simulated characters in VR, personal assistants in MR/AR, robots in physical spaces, and from short horizon, low level to long horizon, high level.
• Enable low-level skills for robotic agent tasks (broad dexterous and functional manipulation, rigid to deformable objects in-hand to against the environment) in a generalizable manner.
• Collaborate with researchers and cross-functional partners including communicating research plans, progress, and results.

Minimum Qualifications:

• Currently has or is in the process of obtaining a Ph.D. degree in Artificial Intelligence, Robotics, or relevant technical field.
• Research experience in embodied AI, robotics, computer vision, machine learning, human-AI interaction, and computer science.
• Experience with RoS, Python, C++, C, or other related language.
• Experience with deep learning frameworks such as PyTorch.
• Experience with 3D experience creation software (Unity, Unreal Engine).
• Experience working and communicating cross functionally in a team environment.
• Interpersonal skills: cross-group and cross-culture collaboration.
• Must obtain work authorization in the country of employment at the time of hire and maintain ongoing work authorization during employment.

Preferred Qualifications:

• Proven track record of achieving significant results as demonstrated by grants, fellowships, patents, as well as first-authored publications on teleoperation at leading robotics (ICRA, IROS, RSS, CoRL) conferences or similar.
• Experience building systems based on machine learning and/or deep learning methods.
• Experience in advancing AI techniques in computer vision, including core contributions to open source libraries and frameworks in computer vision.
• Experience solving analytical problems using quantitative approaches.
• Experience manipulating and analyzing complex, large scale, high-dimensionality data from varying sources.
• Experience in integrating theoretical and empirical research with hardware design and demonstrations to solve problems and demonstrate that solution.
• Experience working and communicating cross functionally in a team environment.
• Intent to return to the degree program after the completion of the internship/co-op.
• Availability for minimum 16 consecutive week internship.

About Meta: