A Forum run by Enthusiasts of MidNite Solar

The Open Source software/hardware corner => The Black Box project => Topic started by: zoneblue on May 02, 2016, 08:02:21 PM

Title: Blackbox on a tablet
Post by: zoneblue on May 02, 2016, 08:02:21 PM
Wanting to have a permanent blackbox display in our kitchen to see whats going on, and to involve, ahem, other members of the household in power use decisions, i originally thought i would use some of the openenergymonitor stuff, particular the GLCD they use. After ordering a couple to try, i just found the whole C coding business incredibly slow going, and finally just bought a cheap 7" tablet for the purpose allowing me to code in php where im quicker.

Android kitkat seemed to work ok for this purpose, the couple minor hurdles were overcome with some free addon apps (Twilight and AutomateIt for the backlight dimming at night, Dolphin for full screen browser). Power consumption is about 1W at night, and about 2W during the day. Using wifi for data, and 5V USB off a dedicated DC converter, in the kitchen wall.

The blackbox module system proved easy to adapt to particular applications and it was simple to throw together a weather forecast module, and ajaxify the entire UI. The home screen power stats update every two seconds, the three weather sources every 3 hours, and the "old" power graphs screen (with its flot based graphs) every 5 minutes. I am finding that information needs vary, and in practice updating the detailed information less frequently works well. The home screen power in and power out is scrolled in real time showing the last two minutes, uses smoothie.js, which seems light and is easy to implement.

(For boB, see how WBJR allows us to access load data... and also here being an example of where id like to see some synchronisation between Icc and Iwbjr, see the negative load blip on the power scroll graph.)

There are a couple of structural changes i want to make to the core code (day roll over logic for tracking SOC and such, storing the last 5 minutes data in more quickly accessible storage), but after that ill be able to finally finish the history UI and other bits of unfinished stuff.
Title: Re: Blackbox on a tablet
Post by: ClassicCrazy on May 03, 2016, 08:33:46 AM
Nice - I wish I knew enough to know how you get the data to display on the Android like that !
Have you tried out Grahams android app ?

Amazon has their 7inch Fire tablet for $40 this week and with careful attention to the process they can be rooted and operating system switched.  I thought about getting another one to do something like what you have there.

Larry
Title: Re: Blackbox on a tablet
Post by: Westbranch on May 03, 2016, 11:47:32 AM
Larry, where does one find out about the procedures to ' root' a device? 

The wife's   e-reader is just laying around as it  is no longer supported  by the maker, they dropped the readers but are  still  around....? Be nice to repurpose it.
Title: Re: Blackbox on a tablet
Post by: ClassicCrazy on May 03, 2016, 05:38:18 PM
Quote from: Westbranch on May 03, 2016, 11:47:32 AM
Larry, where does one find out about the procedures to ' root' a device? 

The wife's   e-reader is just laying around as it  is no longer supported  by the maker, they dropped the readers but are  still  around....? Be nice to repurpose it.

For the Amazon Fire I mentioned look here - also look up your particular device and see if it is supported. Some are easier to root than others. But read to the end of the posts too - sometimes as in this Amazon Fire there are Amazon firmware updates that have made rooting more difficult or impossible.

http://forum.xda-developers.com/amazon-fire/development/amazon-fire-5th-gen-supertool-root-t3272695

also look up your device on youtube to see if someone has done it before - I like teenagers videos because they go through all the details !
Title: Re: Blackbox on a tablet
Post by: zoneblue on May 03, 2016, 10:36:25 PM
Larry, i havent done this using 'apps'. Its just a full screen web browser that gets webpages from the blackbox. So no android hacking required.

Blackbox... thats another matter, sorry but for now mildly serious linux fu required to install it.  The good news is its stable, been running now for over two years. And the classic, is, wahoo, now also stable. So we are good to go.

Have started fiddling with my AXS port, and itll be a straight forward to write a module for it. The main problem there is that the firmware changes outback made, have been pretty significant (for optics), and itll need to cope with that. n1gnn didnt work for this reason, its coded for the pre optics firmware.

Title: Re: Blackbox on a tablet
Post by: Westbranch on May 04, 2016, 12:16:18 PM
ZB, I was just looking up the apps you listed and find there are a swack of 'AutomateIt' and 'Dolphin' Apps to choose from, which ones did you use?

PS, I also had to look up KitKat and found that it is Android 4.4...  for some reason the cutesie nicknames don't get used much over here...??
Title: Re: Blackbox on a tablet
Post by: zoneblue on May 06, 2016, 05:46:52 AM
The lad put Dolphin on my phone last time he was here. See links below.

Of all the browsers i tried it was the only one that natively works with kitkat's "immersive mode", ie hiding the status and navigation bars. It may be that the newer OSs are easier but the cheaper tablets all seem to come with versions of kitkat (4.4.2, 4.4.4)

Automate it, is a task scheduler (cheaper version of tasker).

Trap for the unwary... cheap tablets most often dont have an ambient light sensor, thus otherwise good apps like Lux Lite, and Velis Auto Brightness, dont work. I still havent got this exactly how i want it, the auto dimming thing is still evading me. These things are designed (over designed in this case) to aggressively manage battery power. You can make the screen stay on from a setting in Developer Options. Then use a time of day based brightness control app.

https://play.google.com/store/apps/details?id=mobi.mgeek.TunnyBrowser
https://play.google.com/store/apps/details?id=com.urbandroid.lux
https://play.google.com/store/apps/details?id=AutomateIt.mainPackage
https://play.google.com/store/apps/details?id=com.vitocassisi.luxlite
https://play.google.com/store/apps/details?id=com.velis.auto.brightness

Title: Re: Blackbox on a tablet
Post by: zoneblue on June 09, 2016, 07:24:43 PM
As a bit of an update on this:

1. Auto brightness continues to evade me. Heres what i learnt through trial and error (for kitkat).
- stock auto brightness only works if you have an ambient light sensor
- the manual display brightness control doest seem to effect power consumption, only display dimming.
- dimming cant be manually disabled.
- dimmed brightness isnt effected by manual display brightness. Thus AutomateIt type approaches dont really work.
- the dimming delay period does change when you change the display standby period (with display always on in developer tools).
- i still get flickering when dimmed, and many others report same. Some of the apps allow you to disable dimming.

So for now im using standby period 30mins, and living with the higher power consumption. However I believe that the paid versions of both Lux and Velis auto brightness have the ability to run without ambient sensors. If i get over my reluctance to get a play account, maybe ill try them.

2. Full screen. For such a simple thing this is surprisingly hard to do.
- most android browsers all hide the location bar upon scrolling (if page tall enough)
- all the net language around full screen is mostly talking about hiding the browser location bar, using javascript  window.scroll hacks. Because thats as much as you could hide at the time, they defined full screen as location bar hidden. Thus these approaches fail.
- defining full screen is also complicated by changes made to the android OS in recent years. Its only since later versions of kitkat that the main device navigation bar wasnt part of the device, and thus became hideable. There were always various ways to hide the main status bar, but we dont need to look at that now.
- using the dolphin browser which has a sort of working implementation of immersive mode. It works a bit eratically, and scrolling still reactivates the location bar.
- Dolphin is not terribly stable on my device, crashing at least once a week.
- Chrome is a much bigger application, as is firefox. Firefox for android, on this device crashes with my javascript heavy app within 5 seconds .
- Chrome supports two helpful things which when added together get you a true full screen web app.
a) Install to homescreen (under settings) described here https://developer.chrome.com/multidevice/android/installtohomescreen This disables the location bar, and gets you a home screen shortcut, and fixes the aspect. When you set either "standalone" or "fullscreen" for the "display" attribute in the manifest file, neither triggers immersive mode.
b) a javascript webapp call to requestFullscreen  described here http://stackoverflow.com/questions/9454125/javascript-request-fullscreen-is-unreliable works and ive tested it as working. But the user still has to click something to allow it to work. Google basically refuses to allow immersive to work at the webpages request. I can live with that, and i put ont he onclick on the app home button.

- there is an app called GMD Full screen immersion https://play.google.com/store/apps/details?id=com.gmd.immersive&hl=en which ahem i havent tried, looking as i was for a in browser solution. But it might be worth a shot.

Theres a somewhat of an up to date summary of the full screen issues here http://www.html5rocks.com/en/mobile/fullscreen/

Title: Re: Blackbox on a tablet
Post by: danda on June 09, 2016, 08:33:43 PM
hey thx for the update zoneblue.  As a blackboxproject user, I'm happy to see advancement.

Also, you might be interested to hear that these past two days I've been writing a script called solarvisor (https://github.com/dan-da/solarvisor) that uses the real.php from theblackboxproject as a data source.   In case others might find it useful I started a thread about about it here:

http://kb1uas.com/mnsforum/index.php?topic=3266.0

I used the real.php webservice because theblackboxproject is already up and running and it is on another machine, so that was the easiest and fastest way to get the data.  very handy!
Title: Re: Blackbox on a tablet
Post by: zoneblue on June 09, 2016, 11:24:34 PM
Like a API kind of thing... Why not, awesome.

Theres been a bit of interest in blackbox lately, so expect to see some new code soon.

Regarding your opportunity loads, yes i think thats an area with wide application. Tapping  into SOC and charge stage of course will give you more options, and the extreme ideal is tapping into the controllers native Waste not aux data, and/or using a light sensor to compare what your producing with what you could produce. This latter for PWM type applications (heating water being the obvious one).

But thats getting ahead of ourselves. We are going to focus on these for now:
- faster backend data storage
- graphs, and history
- configurability

Of course your welcome to throw your oar in, even just ideas.


Title: Re: Blackbox on a tablet
Post by: danda on June 10, 2016, 01:32:44 PM
QuoteTheres been a bit of interest in blackbox lately, so expect to see some new code soon.

sweet, glad to hear that.  in my experience, blackbox has the technical underpinnings in place but needs UI cleanup and polish to reach full potential.

Regarding SOC:   I see in real.php at present that SOC is 100%.  Yet it is 10am, charge mode is bulk and voltage is only 55.3, so there's probably 2 hours of bulk left before it hits absorb.   I haven't really paid attention to SOC.  Maybe it is always 100% unless whizbang jr is installed?  Or else can you tell me how to interpret?

I have a separate trimetric current / SOC meter, but it is old school and I have no way to access it over the network unfortunately.

Quotecontrollers native Waste not aux data

I'm not familar with that, though it rings a dim bell.  Can you expand or do you have a pointer to more info?

I don't think I see anything related in output of real.php.   How hard to add?

Classic Unit|150| 
Firmware Date|2014-12-18| 
Lifetime kWh|3601.5|kWh
Uptime|0.44|days
FET Temp|48.4|°C
Efficiency|99|%
Output Power|2657|W
Output Voltage|55.3|V
Output Current|47.8|A
Charge Stage|Bulk| 
Battery Temp|23.2|°C
State of Charge|100|%
Days Since Float|1|days
Max Batt Voltage|55.3|V
Min Battery Voltage|55.3|V
PV Voltage|59.8|V
PV Current|43.9|A









Title: Re: Blackbox on a tablet
Post by: zoneblue on June 12, 2016, 02:19:08 AM
Yes i imagine the classic will not report anything sensible without the WBJr.  You can add the WBJr to your existing shunt, you just need enough copper washers to stack them up with your exisiting monitor. Well worth it for SOC and EA.

Re the classics waste not modes, both aux1 and aux2 have waste not. Basically it turns on the aux port according to weather the classic thinks it has spare power. As far as i can tell its a feedback based thing, that when you draw too much it turns off. AUX2 is PWM around 1kHz, and Aux1 is PWM about 0.01Hz... or something. (theres sensitivty/delay/hold type settings). Bear in mind you need aux2 for WBJr, and Ross's solution looks attractive. Indeed ill be adding something similar to BB here in due course, using a small 0.2W PV as the reference.

Title: Re: Blackbox on a tablet
Post by: zoneblue on June 12, 2016, 02:20:22 AM
Oh theres  other aux modes as well, at least one is SOC based.
Title: Re: Blackbox on a tablet
Post by: danda on June 13, 2016, 05:34:44 PM
ok, yeah now i remember reading about the aux in the manual.  So that's all fine for DC loads near the charge controller.

My solarvisor script is aimed at running (typically) AC loads controlled by a computer in the house or anywhere connected to the inverter.

So the questions that come to mind then:
    is blackbox able to read and report the AUX on/off status? 
    does AUX (waste not) even work without WBjr?   seems like it should turn on in absorb/float.
    would that flag even be useful for AC loads when nothing is connected to AUX?

I don't see an aux field anywhere in the real.php output...

I think SoC plus charge mode and voltage is plenty sufficient anyway.  Just curious really.
Title: Re: Blackbox on a tablet
Post by: zoneblue on June 13, 2016, 07:30:53 PM
Im not really an expert, but what i think is that:
- aux has many modes, some based on voltage, some on charge stage, and some on SOC.
- SOC based ones will need WBJr
- AUX states are accessible over modbus, pretty sure of it.
- that 'real' script was just intended as an example. Using the full module configuration system you can access any modbus register that you need. Admittedly its a bit hard to follow at present, but that will improve.



Title: Re: Blackbox on a tablet
Post by: zoneblue on July 09, 2016, 12:00:33 AM
For the sake of completeness, the Android screen dimming issue can be solved using a deeply buried Dolphin setting.

Settings, advanced, more, Keep screen on.

(dont you just love android menus, yuk)

So to recap:

Android settings
- Developer options, stay awake ( keeps the screen from turning off while plugged in).

Dolphin browser:
- settings, advanced, more, Keep screen on. (stops screen from dimming)
- control panel, full screen (enables immersive mode, ie hides android nav and status bars) OR  javascript request full screen mode (same effect)

Automate It:
- reduce screen brightness (or turn screen off) at 9pm
- increase screen brightness at 7am.

Power consumption seems to follow no discernable pattern from any of the above. Dimmed does seem to result in the lowest consumption, although i also saw it pretty low using one of the screen brightness apps.

Dolphin still crashes from time to time, however Chrome and the others were all worse in that respect. I think theres something about the heavy javascript (ajax, json, graph libs, jquery etc) in there that the browsers struggle with. I might try to reduce the libs a bit, since it appears that flot can do scrolling graphs.