The no-buy year experiment

Lately I’ve been having a debate with myself. In a lot of cases, I feel that I am pretty intentional with the things that I have in my life. In areas like friendships, work or family, I just have what I need. However, and because I have taken up a few hobbies and I had a bigger budget, I have been buying more material stuff. I do not like this, and I would like to change that aspect of my life, or at least improve it. There are multiple books about minimalism, and I have read quite a few of them:

Read More

Ommiting corrupted images inside a Pytorch dataloader

Sometimes our data is not as clean as it should be. Maybe some images are corruped, or the entries inside the database were not as expected. In that case, iterating through them using the Pytorch dataloader can be hustle. Recently I learned that you can clean your data after it has been loaded, but before you are iterating throught in your training lops. In this case I am going to give an example with images, but it can be applied to any kind of data.

Read More

Count NaN values that are not NaN in Pandas DataFrame

I am working with a dataset that contains lots of missing values. However, when a value is missed it is not represented as NaN but as “–”. In order to find how many missing values there are in an specific column I cannot use typical Pandas functions. Henceforth, I have worked it out by count the number of appereances of this character in the column and substract it to the total number of entries in the column:

Read More

Moving from Tensorflow 2.0 to Pytorch

I was having many issues with GPU memory management in a Transfer Learning problem when using Tensorflow 2.0 with Keras module. Even though I was using Inception V3 model with all the weights of the architecture freezed, the code ran out of memory. The weights of the model occupied only 98 MB therefore that wasn’t the problem. Input shape was (299, 299, 3) so not huge images were being used, but I couldn’t use a batch size higher than 16.

Read More