07
Neural Network
Architecture

Neural Networks are a set of nodes that perform non-linear transformations over linear combinations of the input variables. These transformations are not set and they are actually learned during the process, optimising coefficients through an activation function.

These learning layers are in the order of the hundreds between input and output. For the learning process of these nodes a subset of the input data is aimed at training the data, while the remaining data is used to test the performace of the neural network.

Neural Network
Build

In order to implement a neural network we use the Torch Python package. We first perform a normalization of the input data, as neural networks work better when the data is normalized.

We then define the type of neural network we are going to use, the input and output dimensions and the number of hidden layers.

Structuring the data as a proper input for a neural network model implies the slicing of data into batches.

Loss criterion must also be specified for the model as well as the optimizing parameters. Once all of this is done we train the model.

Torch
Tensor

hidden=100

class SimpleNN(nn.Module):
   def __init__(self):
   super(SimpleNN, self).__init__()
   self.fc1 = nn.Linear(5,hidden)
   self.fc2 = nn.Linear(hidden,1)

def forward(self, x):
   x = torch.relu(self.fc1(x))
   x = self.fc2(x)
return x

X_train = torch.tensor(alles).to(torch.float32)
y_train = torch.tensor(lmd_soilWetness).to(torch.float32)
y_train = y_train[...,None]

class MyDataset(Dataset):
   def __init__(self):
   self.data = X_train
   self.labels = y_train

def __len__(self):
   return len(self.data)

def __getitem__(self, idx):
   return self.data[idx], self.labels[idx]

dataset = MyDataset()
dataloader = DataLoader(dataset, batch_size=100, shuffle=True)

model = SimpleNN()
criterion = nn.MSELoss()
optimizer = optim.SGD(model.parameters(), lr=0.1)
iter=0

for epoch in range(100):
   for inputs, labels in dataloader:
      model.train()

      # Forward pass
      outputs = model(inputs)
      loss = criterion(outputs, labels)

      # Backward pass and optimize
      optimizer.zero_grad()
      loss.backward()
      optimizer.step()

Loss
vs Epochs

The training of the neural network model can be monitored looking at the loss of the model at each iteration step. The result of this is the Loss vs. Epoch plot shown here.

The complexity of the input data is reflected on how the model learns and finds out patterns and behaviours that can be parameterized.

Spatial
Distribution

The neural network model captures the overall spatial distribution, with values closer to one in the Equator along South America, Africa and Asia.

On the contrary, it misses the high values in northern Europe and North America, that were better captured by the polynomial regression.

Difference
in Dataproducts

The soil moisture dataproduct derived from using a neural network model has shown the least differences with the LMD soil mositure dataproduct used as a reference.

We can observe how the differences are minimal on a global scale, with small pockets of discrepancies on South and North America of negative and positive sign.