Manually Charging Our US2000C Solar Battery
Earlier this month, I discovered a couple of issues with our Solar setup.
Two apparent bugs had combined and resulted in our solar battery going completely flat: so much so, that the battery couldn't power it's own Battery Management System (BMS).
Without a functional BMS, the inverter won't send a charge to the battery, so it wasn't possible to recover from this situation with the inverter alone.
Instead, I had to climb up to the battery and use a lithium battery charger to charge one of the battery units.
This post talks about _why_ our Pylontech US2000C batteries went dead as well as describing what I did to get them back into service.
* * *
### What Went Wrong?
#### Bug 1: My Stuff
Earlier this year, I switched from using the inverter's static schedule to having Home Assistant dynamically manage our charge schedules.
In my post about that process, I noted that it also involved a change in paradigm:
> It's worth noting, however, that this _does_ come at a small cost, because we've changed paradigm a bit.
>
> Previously, the configured schedule was stored in the inverters registers, so would apply whether or not there was connectivity between the inverter and Soliscloud.
>
> Homeassistant, though, sends an event when something causes it to start - so if connectivity is down, the inverter won't switch the battery over to charging.
Almost inevitably, fate jumped up and bit me on the arse.
* * *
##### It's Always DNS
On the 24th of January, Docker's built in DNS service stopped resolving names, which left the auth proxy container unable to resolve Soliscloud's address:
urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x7f41aca32850>: Failed to resolve 'www.soliscloud.com' ([Errno -3] Try again)
As a result, Home Assistant was unable to tell the inverter to start charging the battery from grid, meaning that the only thing that would charge the battery was excess energy from our solar panels.
Excess solar energy... January in the UK...
* * *
##### Where is Alerting?
By this point, you might be tutting and observing that _maybe_ I should have been monitoring things.
I had monitoring in place but (you guessed it), Grafana was running on the same box and so Docker's breakage had left it unable to send notifications:
logger=ngalert.notifier.alertmanager org=1 t=2025-01-24T15:08:20.074946486Z level=error component=alertmanager orgID=1 component=dispatcher msg="Notify for alerts failed" num_alerts=
1 err="Ben/email[0]: notify retry canceled due to unrecoverable error after 1 attempts: failed to send email: dial tcp: lookup smtp.<redacted>: i/o timeout"
I _might_ have caught the issue myself if I'd looked at solar stats, but given the lack of sunshine, I wasn't really looking.
* * *
#### Bug 2: The Inverter
Although I didn't really write about it in my previous post, the inverter has (what should have been) a safety net.
The inverter settings include an option called `Forcecharge SOC`
If the battery's charge level drops below 18%, the inverter should start charging from grid to bring it back up.
This means that, in theory, even if my charge scheduling failed the battery should always have remained in a serviceable state.
However... it seems that this setting didn't kick in.
* * *
#### The Battery's Death
The battery stopped being charged on 24th of Jan, however it was fine (in a "not completely dead yet" sense) until the early hours of 31st Jan.
Although the reported charge level had been hovering around 20% for days, it plummented overnight and continued to drop during the day:
The Voltage and Current graphs indicate that this wasn't the result of some load suddenly being put onto it.
The spikes in current and voltage align with increases in the reported SoC indicating that the forcecharge _probably_ was kicking in.
Although there's no direct correlation with the drop, temperature graphs show that it got quite cold overnight that night
Whilst the temperature was still well within the stated serviceable range of the battery, it's not the first time that I've had suspicions about these batteries and temperature.
Temperature issues _would_ also explain why the inverter didn't start force-charging: if the battery got cold enough to switch into protection mode, the inverter would not have been _able_ to charge it.
Whether it was the cold, or the battery packs just giving up after a week of not being charged, the battery was now down.
* * *
#### Powering the batteries on
Hoping to be able to convince the inverter to start a charge, I climbed up to the batteries and flipped switches to switch them back on.
As soon as they powered on, the alarm light started to flash before the units eventually powered themselves back off.
During that time though, although it couldn't charge them, the inverter **did** manage to collect stats (I did this a few times for comparison):
The manual for the Pylontechs says that the BMS will not operate if the battery voltage reports below `44.5V`. The highest value on that voltage graph, unfortunately, is `44.1V`.
* * *
### Manually Charging
Given that the batteries were too depleted for the inverter to start charging them, I was left with two options
1. "Jump" them with another battery
2. Plug them into a wall charger
The aim of both is to pull the voltage up just enough that the battery is operable, allowing the inverter to take care of the rest.
If I was a battery technician, I'd probably have a spare battery for option `1`, but (contrary to reports) I'm not.
This meant that I needed to find a way to charge at least one of battery units myself.
Note: I don't know if you've ever googled "how to charge a solar battery", but the results are not _particularly_ helpful for this (answers: use solar, use an inverter etc).
One thing that I **did** know was that it was important to use a charger designed for lithium batteries. Chargers intended for lead-acid batteries tend to pulse, which lithium packs don't like (lithium battery packs are also arsonists: if they don't like something, they tend to try and burn it).
The user manual provided some additional useful information
Item | Value
---|---
Battery Type | LiFePO4
Nominal Voltage | 48V
Charge Voltage | 52.5 ~ 53.5V
After a bit of hunting around I managed to track down and buy a charger.
* * *
#### Making a Lead
The charger that I'd bought came with crocodile clips, but Pylontech batteries have a waterproof shroud around their posts, preventing the clips from connecting onto them.
I found some pylontech to ring connector leads online: these are what are used to connect the battery to an inverter:
I cut the clamps off my charger, stripped the cable back and then wrapped and soldered it onto the ring connectors.
Then I wrapped both in electrical tape, pulling the charger wire up a little to act as a form of strain relief
* * *
#### Connecting The Charger
I climbed a ladder up to the batteries, taking a powder fire extinguisher and one end of an extension lead (the other was still unplugged downstairs) with me.
To make things safe, I isolated the panels
Then I isolated the AC supply into the inverter
Each of the battery units has a power switch on the left hand side:
I switched each of these off.
My aim was to partially charge one of the packs: just enough to pull the voltage up so that the inverter could ultimately take over.
To avoid having to reach over the other packs, I chose the one nearest to me and disconnected the cabling.
Note that the comms cable has also been removed
Then I prepared the charger:
* Plugged its mains lead firmly into the back of it
* Connected its mains lead to the extension lead
* Pushed its negative connector onto the negative post on the right
* Pushed its positive connect onto the positive post on the left (i.e. the one without the cover)
I flipped the battery's power switch back on before pressing the red button for about 2 seconds.
The battery lit up and started flashing it's alarm light. One of the charger's LEDs lit up, indicating that it was correctly connected.
I carefully stepped over it, climbed down the ladder and plugged the other end of the extension lead into the mains.
The charger's second LED lit up to indicate that it was now charging
The battery's alarm light had stopped blinking and `RUN` was now illuminated. The first LED on the charge level indicator was flashing
It was working!
I left the battery to charge for about 90 minutes before checking back in on it. The first charge LED was now solid, with the second flashing.
According to the manual, each LED is "worth" 16% of charge, so I knew that the battery was charged to at least that level.
* * *
#### Switching Back Over
I climbed back down the ladder, switched off (and unplugged) the charger before climbing back up, turning the battery off and then disconnecting the charger from the battery.
Being careful to make sure that each cable went back into the correct place, I connected the unit back to the others.
I flipped the power switch on each and then held the red button down on the master (the one without a cable in link port 0 - the top battery in my case).
One by one, the units powered on. The first two very briefly flashed an alarm light, but then their `RUN` light illuminated
The battery was online!
Knowing that the charge may not last long, I quickly edited the inverter's settings to trigger a grid charge.
The inverter kicked the charge in immediately:
I let the charge continue until the battery was reporting around a 30% charge.
After I switched the charge off, the batteries began to supply the house until they reached 20% and discharging stopped:
I did this partial charge & discharge for a couple of reasons
1. I wanted to be sure that the battery **did** stop at 20%
2. Battery balancing: the manually charged battery was slightly more charged than the others - I reasoned that discharging would probably help even this back out
With the battery appearing to be working correctly, I left the inverter to kick the overnight charge in automatically (good timing too, we've _finally_ had some cheap leccy prices again!).
Checking the graph the next morning showed that things had behaved as expected
* * *
### Avoiding Repeat Incidents
Once I realised the battery was dead _and why_ , my initial reaction was to want to turn all of the smarts back off and accept occasionally spending a couple of pence extra.
**But** , that was almost certainly an over-reaction because the issue can just as easily be mitigated by
* Having a once daily charge configured on the inverter
* Turning alarm notifications on in Soliscloud
* Reconfiguring containers not to use Dockers DNS service
I did also toy with the idea of having a script poll Grafana's API to list active alerts, but that would have created a lot of routine noise, driving alert fatigue.
* * *
#### Conclusion
Urgg, just urgg... although the inverter's forcecharge setting _should_ have saved me, this was my own fault really: I'd _even_ identified that the change of paradigm brought some risk, but hadn't considered some of the more extreme outcomes.
The fact that I've had to buy a charger (and leads) means that the scheduling changes I made will now very likely never break even (though we shall have to see).
Given that this is the second time I've suspected that low temperatures had a hand in issues with our battery, I'm going to run a temperature probe up to the batteries themselves so that there are more granular stats the next time that there's an issue.
It does seem though, that fate is determined that I should one day become a battery technician. But, I'm not ready to cell out yet.