Recently, a machine learning approach to Monte-Carlo simulations called Neural Markov Chain Monte Carlo (NMCMC) is gaining traction. In its most popular form, it uses neural networks to construct normalizing flows which are then trained to approximate the desired target distribution. In this contribution, we present a new gradient estimator for the Stochastic Gradient Descent algorithm (and the corresponding PyTorch implementation) and show that it leads to better training results for the model. For this model, our estimator achieves the same precision in approximately half of the time needed in the standard approach and ultimately provides better estimates of the free energy. We attribute this effect to the lower variance of the new estimator. In contrary to the standard learning algorithm, our approach does not require estimation of the action gradient with respect to the fields, thus has the potential of further speeding up the training for models with more complicated actions.