Hi, my name is Jack Rogers
I'm an aspiring Software Engineer.

This capstone project focused on the creation of custom prediction models for autonomous driving, done in collaboration with Northrop Grumman. I led the development of a polished and user-friendly web app that hosted and demonstrated our team's prediction models. The application, developed using HTML, CSS, Sass, and JavaScript, served as an interactive interface that significantly impressed our client, Northrop Grumman.

Furthermore, I assumed the responsibility of the communications lead, ensuring smooth collaboration and timely project delivery by facilitating effective communication between my team and Northrop Grumman.

One of the primary technical achievements of this project was the design and implementation of highly accurate and robust prediction models using Python, PyTorch, and L5Kit. The precision and performance of these models not only met but surpassed the client's requirements, earning high praise for their technical performance.

Notably, the entire web app showcasing our project's accomplishments was my creation, highlighting my ability to merge technical understanding with user-centric design.

The main focus of this project was to implement value iteration and Q-learning, two crucial concepts in reinforcement learning. The algorithms were initially tested on Gridworld, a standard test environment used in reinforcement learning, to ascertain their correctness.

The project then expanded to apply these algorithms to a simulated robot controller, referred to as Crawler, and the classic arcade game, Pac-Man. Both these applications offered a more complex and realistic environment, testing the robustness and flexibility of the implemented algorithms.

A significant part of the project was dedicated to coding the value iteration and Q-learning agents for Gridworld, Crawler, and Pac-Man in Python. The code was tested for technical correctness using an autograder, which ensured that the algorithms behaved as expected in various scenarios.

The project also involved analyzing and answering specific questions related to the behavior of the agents and the learning process. This aspect was crucial in understanding the theoretical underpinnings of the implemented algorithms.

This project reinforced my abilities in reinforcement learning, particularly in implementing and understanding the intricacies of value iteration and Q-learning. It also displayed my capacity to apply these techniques to solve complex problems in various scenarios, ranging from grid environments to game domains and robotic controls.

The main objective of this project was to implement "curve fitting" or regression for given data using genetic algorithms (evolutionary search). The project was conducted by running various benchmark examples to study its performance.

The crux of the project was the regression problem, a fundamental problem in machine learning, where the task was to find a function 𝑦 = 𝑓(𝑥) that passes through some given data points. Given the challenge of overfitting in machine learning, the aim was to find a function that minimized the loss function or provided a least square fit.

One of the main challenges faced was the unknown form of the function 𝑓, thus requiring a search for 𝑓 from a family of functions {𝑓1, …, 𝑓𝑁}. This aspect introduced the complexity of dealing with a potentially infinite set of functions.

An important component of the project was generating arbitrary expressions at random, done using the generate_random_expr function in the makeRandomExpressions.py file. This function was crucial for creating diverse and random expressions for the genetic algorithm to work upon.

Additionally, a viability check for each expression was also implemented to eliminate any non-viable or ill-defined expressions. This was achieved using the function is_viable_expr() in the file fitnessAndValidityFunctions.py.

A key aspect of this project was the use of a fitness function to calculate the negation of the least square error, with the goal of maximizing it. This function, compute_fitness(), was crucial in framing the genetic algorithm in terms of fitness maximization.

Lastly, the project required the implementation of two essential operators for genetic algorithms: crossover and mutation. These operators, random_subtree_crossover() and random_expression_mutation(), respectively, allowed the algorithm to generate diverse solutions and traverse the search space effectively.