Which loss function to choose for my encoder-decoder in PyTorch?

Question

I am trying to create an encoder-decoder-model, which encodes an 10x10 list and should decode it to an 3x8x8 array/list. Which loss function should I choose to achieve this? I know that the shapes of the input and output are very random and I'm not quite sure how to even fit both into one encoder-decoder-model. The 3x8x8 output however is mandatory and the 10x10 shape is the difference between two nested lists.

From what I have researched so far, the loss functions need (somewhat of) the same shapes for prediction and target. Now I don't know which one to take, to fit my awkward shape requirements.

Please provide enough code so others can better understand or reproduce the problem. — Community
– Community Bot, Commented Apr 12, 2023 at 15:31

tgaudier · Accepted Answer · 2023-04-12 14:24:08Z

The choice of your loss function should be guided by the task you are trying to perform, and not necessarily by the shape of your input/output. Here are some examples of usual tasks and frequently used losses :

If you are doing classification (predict one class among n possibilities), Cross-Entropy could be interesting.
If you are doing regression (predict values on continuous axes), Mean Squared Error could be well suited.
If you are trying to predict a probability distribution, you could use Kullback-Lieber divergence or log-likelihood.
Multiple other losses exists (see this page from pytorch doc), some are specific cases of other. You can also build your own loss if none suits your problem.

To sum up, we would need more information about what is your output (images, sound, classes, position prediction, text tokens...) to tell which loss is the best for your model.

Collectives™ on Stack Overflow

Which loss function to choose for my encoder-decoder in PyTorch?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related