HW · ME 595R

HW 1: Vanilla NN

due 1/16/2025 before midnight via Learning Suite 25 possible points

Download the Auto MPG dataset, specifically the file auto-mpg.data. The auto-mpg.names file describes each of the 9 columns. Our goal will be to use this data to predict an automobile’s mpg as a function of the other parameters (except for “car name”, which we won’t need).

First, some data preparation. Some of the rows have missing values —eliminate those rows from your dataset (you can do that beforehand or just in a loop when you read in the file). Next, normalize the inputs of each column using a standard normal distribution:

\[\hat{x_i} = \frac{x_i - \mu_{x_i}}{\sigma_{x_i}}\]

where \(\mu\) and \(\sigma\) are the mean and standard deviation respectively of the column. It is generally desirable for the input data to be zero-centered, and also helps avoid biasing weights towards a particular sign. It is also usually helpful to normalize so that we don’t bias the influence of some parameters over others just because of their magnitude (i.e., a unit choice). It is often also helpful to normalize the targets (mpg), though if you do, remember to unnormalize them when plotting/printing results to get back the actual mpg values.

Finally, randomly separate the data into a training set and a testing set (with an 80/20 train/test split).

Setup a neural net. You can use however many layers you want, but for a problem this size you should be able to do well with three layers (two hidden layers). You’ll need to experiment some with different layer widths, activation functions, batch sizes, learning rates, and number of epochs. You should be able to an average absolute error of under 2 mpg on the test set. You’ll see some scatter in the results for sure, but that’s still pretty decent predictive capability given how little data we have (and no physics).

To turn in:

Plot the objective (mean squared error) across each iteration for both the training and testing set on the same plot.
Once training is complete, report your average absolute error of the test set in mpg (again you should be able to get under 2, but we’ll say under 2.5 to give some buffer).

Not required stuff:

Note that while MSE is a reasonable objective, it is not always the most intuitive to interpret so it can be helpful to plot other quantities (like in this case mean absolute error).
One way to look at predictive capabilities is to plot the actual mpg and the predictions, both from test set, against each other. Plot also a straight diagonal line for reference (the line of perfect predictions).