Anyway since it was already broken i could just as well hook it up to a power supply. If you just force the battery casing upwards, it will create some room between the contacts on the print. Then just move it up and down till they break

Then just find yourself a power supply for it. Since its around 4.5v and my garage is filled with power supply's i went with the easy one. Find a power supply that has a USB connector on the end. Since usb is +- 5 volt it will work just fine (think of those jabra headsets/nokia phone with micro usb etc etc) anyway just read on the print to see if its around that voltage.
To keep it easy i made a picture also showing which is plus and minus. Maybe handy for someone.
Yes i could have done it a bit better, but it works just fine.