Autonomous Driving Simulator

Develop an interactive 3D autonomous-driving simulation using a game development engine such as Unity, Unreal Engine, or Godot. The simulation will feature realistic vehicle physics, a configurable road or racing environment, virtual cameras, and simulated LiDAR using raycasting. Students will implement autonomous driving behavior that allows a vehicle to navigate the environment, recognize or detect obstacles, and complete a series of progressively more difficult driving missions.

The project will explore how autonomous vehicles use simulated sensor information to interact with dynamic environments. Missions may progress from basic vehicle control and route following to obstacle avoidance, navigation through increasingly complex environments, and handling moving obstacles. Depending on team experience and available computing resources, additional features may include integrating a pretrained local AI model for camera-based object recognition and comparing camera-based perception with simulated LiDAR measurements.

The goal is to produce a functional, interactive simulation that demonstrates autonomous navigation, virtual sensing, and measurable driving performance. The project should prioritize reliable functionality and achievable milestones over photorealistic graphics or training large machine-learning models from scratch.

Developing and evaluating autonomous-driving systems in real-world environments can be expensive, technically challenging, and potentially unsafe during early experimentation. Students and developers need accessible simulation environments in which they can explore vehicle control, virtual sensors, perception, and autonomous navigation without requiring a physical vehicle or specialized sensor hardware.

Existing game engines provide vehicle physics, rendering, and other simulation capabilities, but integrating these components into a functional autonomous-driving system with clear objectives and evaluation metrics requires additional software development.

This project addresses that challenge by creating an accessible, mission-based autonomous-driving simulator in which users can observe vehicle behavior, visualize virtual sensor information, and evaluate autonomous performance across progressively more difficult driving scenarios.

By the end of the Capstone sequence, the team should deliver a functional autonomous-driving simulation with the following components:

Core Deliverables

• A working 3D driving environment developed in Unity, Unreal Engine, Godot, or another suitable game engine.
• A controllable vehicle with functional physics, steering, acceleration, and braking.
• Simulated LiDAR using raycasting to estimate distances to surrounding objects.
• One or more virtual cameras providing visual observations of the environment.
• Basic autonomous driving functionality, including navigation along a predefined route and obstacle detection or avoidance.
• A series of progressively more difficult driving missions with defined completion criteria.
• A user interface displaying mission objectives, vehicle status, and relevant sensor information.
• Performance measurements such as mission completion, completion time, collisions, and navigation success.
• Source code, setup instructions, technical documentation, and a final demonstration.

Proposed Mission Progression

1. Basic Vehicle Control: Navigate a simple track or road while maintaining control and reaching a destination.
2. Route Following: Follow a defined driving route containing turns and checkpoints.
3. Static Obstacle Avoidance: Detect and navigate around stationary obstacles using virtual sensor information.
4. Complex Navigation: Complete a more challenging course with multiple turns, obstacles, and constrained driving areas.
5. Dynamic Environment: Respond to moving obstacles or vehicles while completing a defined route.

The team may adjust the mission progression based on available resources and implementation complexity.

Optional Stretch Goals

• Integrate a pretrained local computer-vision model for object recognition.
• Compare camera-based perception with simulated LiDAR measurements.
• Add more complex traffic scenarios, environmental conditions, or autonomous driving strategies.
• Provide additional visualization and performance-analysis tools.

The core simulator and mission system should remain functional even if optional machine-learning features are not implemented.

Students should have a working knowledge of programming, object-oriented software development, debugging, and collaborative software engineering. Experience with Python, C#, C++, or a comparable programming language is expected.

Students should be comfortable learning a game development engine, working with unfamiliar libraries or APIs, and dividing a larger software project into manageable components.

Familiarity with basic linear algebra, vectors, coordinate systems, and fundamental physics concepts will be useful for understanding vehicle movement and virtual sensors. Prior experience with autonomous driving, machine learning, or game development is not required, but students should be interested in learning these topics.

At least one team member should have access to a computer capable of running a modern 3D game engine, preferably with a dedicated GPU, sufficient RAM, and a capable CPU. The team should select an engine and configure simulation complexity according to available hardware.

Helpful experience includes:

• Unity, Unreal Engine, Godot, or other 3D development environments.
• C#, C++, or Python programming.
• Computer graphics, 3D coordinate transformations, and physics simulation.
• Vehicle dynamics, navigation, or autonomous systems.
• Raycasting and virtual sensor simulation.
• Computer vision, object detection, and machine learning.
• PyTorch, OpenCV, or pretrained vision models.
• GPU-based development and performance optimization.
• User-interface development and data visualization.
• Git, GitHub, testing, and collaborative project management.

Students do not need prior experience in all these areas. The project is intended to provide opportunities to learn simulation, sensing, and autonomous control through hands-on development.