WiFi board local server + static IP, ideas?

[Sid Genetry Solar]

I was recently talking with a Genetry Solar customer, and they posed an interesting request that I'm certain will be of quite some interest to customers.  I'd like to gather some ideas/interest so it's of use to as many customers as possible.  (I know other inverter manufacturers specialize in telling customers what can't be done...so that's where we at Genetry differ!)

Basically, when the WiFi board is connected to a WiFi hotspot or network of other sort...I would need to add a setting for a static IP address (and of course, a way to see said IP address on the screen).  This way, you could easily hardcode an ESP8266/ESP32 or other WiFi-enabled device to connect to the IP address (on the same local network), and grab data directly from the WiFi board (GS inverter), without using a cloud server as a relay.  For that matter, you wouldn't even need to have Internet connectivity on the WiFi hotspot.

If that's all good, then there's more questions:

• Reading data from the inverter, what would be the best format?
  • I'm considering something like "192.160.0.201/stats.json"--read voltages, temperatures, fan status, etc., etc.--much like what's currently available on the MQTT server
  • possibly other files such as "192.160.0.201/scope_out.txt" for the AC output 'scope datapoints (and "scope_in.txt") for the AC input scope datapoints
    • said files would contain a list of 200 numbers, separated by CRLFs (i.e. separate lines).
    • or is JSON format easier to parse?  (Sure is a heap more work for anything except computers with gobs of memory and a dozen abstraction layers!)
• Any suggestions for "commanding" the inverter?
  • I'm currently considering an HTTP POST request to "192.160.0.201/command" with the command in the POST datastream.
  • I still do NOT plan on opening up system config (i.e. battery setpoints, system mode, etc.) to any HTTP/WiFi access, simply for security reasons.  You're welcome to provide your input/thoughts on this as well.

 

Just polling for ideas, I suppose.

[The Blind Wolf]

Well,.

 

I'm all for html, with each number with a name.

 

such as

 

Input DC Voltage: 27.56

 

Hz 60.1

 

I still want a way to change config remotely from my pc or mobil device.  Maybe have a way to place a password requiment to enable change mode?  Or maybe a one time token such as if you log into the GS inverter, and you want to modify anything you pysically have to go touch each button on the gs inverter to enable it?

 

At least have where I can change some things, anything that is power critcal such as the AC voltage output and Hz should be only allow on the GS inverter it self but, some things should at least be open such as starting a generator, changing the setting for battery max and min.

 

 

[Sid Genetry Solar]

Obviously, I can add an HTML viewer page, but it'll simply read the JSON file and translate it to user text.  This is more for system integration.

[kazetsukai]

Static IP is one option. Have you considered using MDNS to advertise services/name of the GS inverter? Then you don't need to hard code anything, rather look for devices with matching advertised services/ name.

 

I think a token requiring physical access to the inverter would be most secure. You'd have to manage tokens in the wifi board UI somehow (perhaps some allowing read, others config/state changes). Any token request really requires SSL/TLS to be secure though.

JSON is by far the easiest format to parse in any high level computer language.

 

I can help you formalize the API so that it is actually codified somewhere rather than just suggestions/implementation if you'd like. You guys consider opening a Genetry Solar GitHub account? I could contribute code there, like OpenAPI specification documents.

 

Edited April 5, 2021 by kazetsukai

[Sid Genetry Solar]

3 hours ago, kazetsukai said:

Have you considered using MDNS to advertise services/name of the GS inverter? Then you don't need to hard code anything, rather look for devices with matching advertised services/ name.

Guess I haven't.  How feasible would it be to interface with an Arduino?  I know almost anything is possible with an embedded system--but what about a small processor?

3 hours ago, kazetsukai said:

I can help you formalize the API so that it is actually codified somewhere rather than just suggestions/implementation if you'd like.

Definitely need something of that sort I suppose.  I suppose I'm asking for suggestions just to figure out the best way to implement it.

 

3 hours ago, kazetsukai said:

Any token request really requires SSL/TLS to be secure though.

Groan...lots more memory, plus security certificates...providing certs that can be updated, etc.  Local network stuff like I have in mind shouldn't be a huge security risk--maybe?

[kazetsukai]

2 hours ago, Sid Genetry Solar said:

Guess I haven't.  How feasible would it be to interface with an Arduino?  I know almost anything is possible with an embedded system--but what about a small processor?

https://github.com/arduino-libraries/ArduinoMDNS

If the board can support UDP it can support MDNS, both registering services and discovery. 

2 hours ago, Sid Genetry Solar said:

Definitely need something of that sort I suppose.  I suppose I'm asking for suggestions just to figure out the best way to implement it.

If you'd like, we can sync up on Skype about it. I can write a mock web service in about an hour that can act as an emulator, unit test actual inverters and generate OpenAPI documentation, which can be used to generate API code stubs in quite a few languages.

2 hours ago, Sid Genetry Solar said:

Groan...lots more memory, plus security certificates...providing certs that can be updated, etc.  Local network stuff like I have in mind shouldn't be a huge security risk--maybe?

If you expose inverter commands to the local network, that's a fairly huge security risk. Without TLS, anyone sniffing traffic can see your token, get inverter status and issue commands. At the very least I wouldn't allow the user to issue commands to the inverter without TLS. This is a fairly difficult issue to tackle...

What if the user had to POST a certificate to the server before they could call the token endpoint? Like the token process, the WiFi board could block the cert assignment request until the user physically interacts with the inverter to confirm the receipt of the cert. 

 

 

Edited April 6, 2021 by kazetsukai

[The Blind Wolf]

1 hour ago, kazetsukai said:

https://github.com/arduino-libraries/ArduinoMDNS

If the board can support UDP it can support MDNS, both registering services and discovery. 

If you'd like, we can sync up on Skype about it. I can write a mock web service in about an hour that can act as an emulator, unit test actual inverters and generate OpenAPI documentation, which can be used to generate API code stubs in quite a few languages.

If you expose inverter commands to the local network, that's a fairly huge security risk. Without TLS, anyone sniffing traffic can see your token, get inverter status and issue commands. At the very least I wouldn't allow the user to issue commands to the inverter without TLS. This is a fairly difficult issue to tackle...

What if the user had to POST a certificate to the server before they could call the token endpoint? Like the token process, the WiFi board could block the cert assignment request until the user physically interacts with the inverter to confirm the receipt of the cert. 

 

 

Yep, the token thing is what I mention before, you have to tap a button or something to allow for remote chaning of the GS settings.  Though the tocken in fairness could be setup to expir 24 hours that way your not having to keep redoing it over and over.

[TheButcher]

Being realistic, if anyone is in a position to install a packet sniffer on your network at all, let alone in a location central enough to capture the traffic, switches being switches and directing traffic to only relevant ports, it's game over anyway.

[kazetsukai]

12 hours ago, TheButcher said:

Being realistic, if anyone is in a position to install a packet sniffer on your network at all, let alone in a location central enough to capture the traffic, switches being switches and directing traffic to only relevant ports, it's game over anyway.

I absolutely disagree. The assumption on the internet is that your traffic is being monitored, thus TLS/SSL are being used _everywhere_, _all the time_, for both business transactions and otherwise. With WiFi security, simply monitoring the signal transmissions reveals nothing about the content as the link layer contains encryption. However even if for instance someone breaches your WiFi network's password, they still cannot monitor the content of traffic that is transport layer encrypted.

It is my opinion that transport layer encryption is essential if the plan is to expose web endpoints on the device itself for the purpose of controlling / monitoring the inverter. 

12 hours ago, The Blind Wolf said:

Yep, the token thing is what I mention before, you have to tap a button or something to allow for remote chaning of the GS settings.  Though the tocken in fairness could be setup to expir 24 hours that way your not having to keep redoing it over and over.

What I'm suggesting is long lived (potentially indefinitely) tokens granted over TLS, and for the same authorization mechanism (a button pressed on the inverter) to be applied both for token grant (which much happen over TLS) and a POST of the certificate (which does not happen over TLS).

Genetry could ship a self-signed cert on the WiFi board firmware that is the default, users could choose to trust that cert or overwrite it with their own.

 

[TheButcher]

As I said before, if someone has gained that level of access to your network, in this case as you have suggested they have access to your WiFi, it's game over anyway.  If nothing else they now have the ability to take advantage of any number of documented vulnerabilities in various equipment that can be found on home networks these days.  If you are piping control for the inverter over the internet it goes without saying that the traffic should be encrypted, but that was not mentioned before, only exposure to a private LAN.

[kazetsukai]

1 hour ago, TheButcher said:

As I said before, if someone has gained that level of access to your network, in this case as you have suggested they have access to your WiFi, it's game over anyway.  If nothing else they now have the ability to take advantage of any number of documented vulnerabilities in various equipment that can be found on home networks these days.  If you are piping control for the inverter over the internet it goes without saying that the traffic should be encrypted, but that was not mentioned before, only exposure to a private LAN.

The forums' first flame war has erupted! (just kidding)

This is saying "its okay if the inverter is insecure over LAN because other stuff on your network is also likely to be insecure", which is precisely NOT what I was saying. Additionally being able to discover my printer and print to it without my authorization is a different animal than being able to change inverter states. I was saying that even if link-layer security fails (wifi password, etc), people still have transport level security that prevents traffic content from being sniffed. Exposing the inverter over the internet is also a bad idea even with TLS, I wouldn't do it.

Now, it could be that the limitations of the WiFi board are what they are, encryption for this is too much, etc. That's certainly plausible. It may be that dealing with certificates/etc just makes implementing this impractical or the effort/reward isn't that, or a variety of other reasons this cannot be done. I'm just suggesting ways it could in theory be done. Its not the end of the world in this case- without TLS I'd put the inverter on an adhoc network with a device (Raspberry Pi) that acts as a reverse proxy and adds TLS to it all.

[Sid Genetry Solar]

6 hours ago, kazetsukai said:

It is my opinion that transport layer encryption is essential if the plan is to expose web endpoints on the device itself for the purpose of controlling / monitoring the inverter. 

Not my immediate plan; the local server idea was solely intended for access via local network, NOT the Internet.

6 hours ago, kazetsukai said:

What I'm suggesting is long lived (potentially indefinitely) tokens granted over TLS, and for the same authorization mechanism (a button pressed on the inverter) to be applied both for token grant (which much happen over TLS) and a POST of the certificate (which does not happen over TLS).

Genetry could ship a self-signed cert on the WiFi board firmware that is the default, users could choose to trust that cert or overwrite it with their own.

Groan.  Easy to say when millions of bytes of storage are readily available...not so much on a tiny embedded system.  Not saying it isn't possible, just it's not at all easy.

With what limited understanding I have of security certificates, they require the date & time to be known in order to be validated (or invalidated).  There is no RTC (real time clock) available on the entire GS inverter, meaning that time would have to be acquired from an Internet server--or else another big chunk of "security" is invalidated.

[kazetsukai]

20 minutes ago, Sid Genetry Solar said:

Not my immediate plan; the local server idea was solely intended for access via local network, NOT the Internet.

I'm thinking its essential for local network access, without it tokens/authentication is meaningless. TLS or not I wouldn't expose the inverter endpoints directly over the internet.

20 minutes ago, Sid Genetry Solar said:

Groan.  Easy to say when millions of bytes of storage are readily available...not so much on a tiny embedded system.  Not saying it isn't possible, just it's not at all easy.

Right, totally onboard with this.

20 minutes ago, Sid Genetry Solar said:

With what limited understanding I have of security certificates, they require the date & time to be known in order to be validated (or invalidated).  There is no RTC (real time clock) available on the entire GS inverter, meaning that time would have to be acquired from an Internet server--or else another big chunk of "security" is invalidated.

I see no need to validate the certificate at all on the server, the clients are going to decide whether or not its valid on their own. and for any self-signed certificate the trust chain is going to be broken by default. The certificate exists to establish a chain of trust as to prevent someone from spoofing your service. genetrysolar.com's certificate chain prevents an attacker from pretending to be genetrysolar.com through the private/public key mechanism and trust chain. In the case of the inverter, most people I'd imagine would be using self-signed certificates where there is no root trust authority, with certificate expirys set well into the future as to prevent headache.

Edited April 7, 2021 by kazetsukai

[AquaticsLive]

I may be a bit behind on this one but reading Sids first post and thinking a bit.  So as I think this through like we are talking about a regular ESP board.  So a spot to give the Board a static IP sounds like a good idea can feed that in as variable multiple ways.  The MQTT command interface or via the GUI.

There is a bunch of code out there for the web server side of it.  Could be setup fairly basic.  I am thinking the ID number of the unit like PJ_123456789 so something like http://Genetry.PJ_123456789  Now this is more security by obscurity but gives people a bit more work to figure out and the reality its a private network so as long as your not doing port forwarding to the outside WAN should be fairly safe.

The data output model sticking with a JSON makes the data packets small and easy to decode.  Could have a local page to download and configure if you just want html and javascript building a small HTA application is fairly low tech and easy for beginners.  I personally would write a .net project and use JSON dynamic objects to deserialize the data and make a nice little dashboard for my PC.  I would be willing to share anything that I build.  I am sure there may be other people that will make better pages than mine but guessing they would share too would be neat to see other peoples projects.

Having any type of data locally sounds like a great idea.  I do like the MQTT server, but yeah there isn't always internet so having a local network option is cool.

[Sid Genetry Solar]

18 minutes ago, AquaticsLive said:

There is a bunch of code out there for the web server side of it.  Could be setup fairly basic.  I am thinking the ID number of the unit like PJ_123456789 so something like http://Genetry.PJ_123456789

I like how you think 😉.  Wouldn't this remove the need for a static IP address?

19 minutes ago, AquaticsLive said:

The data output model sticking with a JSON makes the data packets small and easy to decode.

Considering "http://genetry.gs39B3D0FEE2/stats.json"...every time you read it, the WiFi board dynamically generates the JSON file.

 

20 minutes ago, AquaticsLive said:

Could have a local page to download and configure if you just want html and javascript building a small HTA application is fairly low tech and easy for beginners.

Also could add on "http://genetry.gs39B3D0FEE2/status.htm" for a quick GUI status page--this a static HTML page that would read the above JSON file and give a page that is easy to pull up from a smartphone, etc.

I really don't want to make inverter settings accessible from any web interface simply due to the security/safety risks.  Best case scenario I would have a setting on the WiFi board similar to the MQTT settings: OFF, R/W and Read-Only.

[The Blind Wolf]

On 4/6/2021 at 7:45 PM, Sid Genetry Solar said:

Not my immediate plan; the local server idea was solely intended for access via local network, NOT the Internet.

Groan.  Easy to say when millions of bytes of storage are readily available...not so much on a tiny embedded system.  Not saying it isn't possible, just it's not at all easy.

With what limited understanding I have of security certificates, they require the date & time to be known in order to be validated (or invalidated).  There is no RTC (real time clock) available on the entire GS inverter, meaning that time would have to be acquired from an Internet server--or else another big chunk of "security" is invalidated.

How easy it it for you Sid to imploment a password system?  say I get my inverter, want to remote into it, I would have to go to the inverter it self, go into remoteing, enable, put in a password, then go to my pc and enter in the IP of the gs inverter, a login pops up sort of like connecting to a ftp and type in a user name that maybe given to the user when they purchased, and then the user have to type in the password they put in the gs inveter to be able to remote into it from the pc.

 

Same thing I do with my NAs on my network.

[AquaticsLive]

11 hours ago, Sid Genetry Solar said:

I like how you think 😉.  Wouldn't this remove the need for a static IP address?

Yeah really do not need the static IP if you got this route it makes it more optional.

 

11 hours ago, Sid Genetry Solar said:

Considering "http://genetry.gs39B3D0FEE2/stats.json"...every time you read it, the WiFi board dynamically generates the JSON file.

Yeah that would work make using it super easy and straight forward.  Even without an interface you can troubleshoot it seeing the pull so building an interface would be much quicker.

11 hours ago, Sid Genetry Solar said:

Also could add on "http://genetry.gs39B3D0FEE2/status.htm" for a quick GUI status page--this a static HTML page that would read the above JSON file and give a page that is easy to pull up from a smartphone, etc.

I really don't want to make inverter settings accessible from any web interface simply due to the security/safety risks.  Best case scenario I would have a setting on the WiFi board similar to the MQTT settings: OFF, R/W and Read-Only.

Yes, I don't think there is a need for command access makes it safer and easier to code you just outputting the variables that you already have.  A short easy page on the status.htm and more in depth from the stats.json.  That keeps it rock solid and secure no way to hack (would be family/kids goofing around) in to do something bad if nothing exists that would permit that to happen.

[The Blind Wolf]

Status is all good, but for those like my self not able to change settings on the GS inverter is no fun.  Ohy well.

[Sid Genetry Solar]

2 hours ago, The Blind Wolf said:

Status is all good, but for those like my self not able to change settings on the GS inverter is no fun.  Ohy well.

Oh I haven't forgotten about you 😉.  Still thinking of making it easy to access a "setup" page by holding ENTER down for 10 seconds when you turn the inverter on--it'll generate a hotspot that you connect to (only 1 device allowed to access the hotspot for security), and an inverter configuration page could be provided there.

But I need to start soldering balancers together, then working on the new code for them...

[The Blind Wolf]

Have you got my balancer made and sent to Sean?  I might as well cash app it to you to get it quicker 😛

[Sid Genetry Solar]

3 minutes ago, The Blind Wolf said:

Have you got my balancer made and sent to Sean?  I might as well cash app it to you to get it quicker 😛

I can't do everything instantly.  I still need to update the code to add the serial communication functions...THEN start hand-soldering each balancer together.

Looks like FedEx purgatory on the remaining parts I need to assemble the balancers...ends on Monday (estimated delivery on April 12th).  Then I have to solder up the whole batch of 400 balancers...then program and test each one--and let's not mention that I'm working a full-time job on house construction in the meantime to pay the bills.  And I have personal things that have to be taken care of as well.

[The Blind Wolf]

3 hours ago, Sid Genetry Solar said:

I can't do everything instantly.  I still need to update the code to add the serial communication functions...THEN start hand-soldering each balancer together.

Looks like FedEx purgatory on the remaining parts I need to assemble the balancers...ends on Monday (estimated delivery on April 12th).  Then I have to solder up the whole batch of 400 balancers...then program and test each one--and let's not mention that I'm working a full-time job on house construction in the meantime to pay the bills.  And I have personal things that have to be taken care of as well.

Poor, Poor Sid, Over worked :P.  Sean got another year and half before it will all smooth out lol.

[Sid Genetry Solar]

UPDATE: I put some code together implementing mDNS and a local webserver to host a dynamically generated JSON file, and a simple webpage (to read said file)...and it worked perfectly the first time.

You can change the mDNS name to pretty much anything you want from the WiFi config page.  (This will all be coming out in firmware version 1.1r3.)

 

Only part I haven't fully worked out is how to control the inverter from the local server (on/off/config, etc.)  It is very easy to handle HTTP POST (form submit, etc.) requests, though I'm not sure what's going to be ideal.

[Waterman]

On 5/28/2021 at 2:07 PM, Sid Genetry Solar said:

UPDATE: I put some code together implementing mDNS and a local webserver to host a dynamically generated JSON file, and a simple webpage (to read said file)...and it worked perfectly the first time.

You can change the mDNS name to pretty much anything you want from the WiFi config page.  (This will all be coming out in firmware version 1.1r3.)

 

Only part I haven't fully worked out is how to control the inverter from the local server (on/off/config, etc.)  It is very easy to handle HTTP POST (form submit, etc.) requests, though I'm not sure what's going to be ideal.

While you are at it, make it so that it can have its own Starlink account. 😉