Software to capture and plot with the new firmwares

robocog
Posts: 64
Joined: Tue Sep 12, 2017 10:06 am

Re: New firmware and serial data capture to visualize and pl

Post by robocog »

The Python code that does the stripping of the header and replaces the commas with tabs :

#open data.txt generated by read serial
#read the first 18 lines and write to a file called header.txt
f = open("all_data.txt", "r")
header = open("header.txt" , "w")
for i in range(1037,1055, 1):
data = f.readline()
header.write(str(data))
header.close()
f.close()


# stripping the first 18 lines of stuff from all_data.txt we dont want nor need
with open('all_data.txt', 'r') as fin:
data = fin.read().splitlines(True)
with open('just_the_data.txt', 'w') as fout:
fout.writelines(data[18:])


# remove commas and replace with tabs so file is ready for use with gnuplot
with open('just_the_data.txt') as fin, open('gnuplot.txt', 'w') as fout:
for line in fin:
fout.write(line.replace(',', '\t'))


# clean up after the process
# remove just_the_data.txt file
import os
os.remove("just_the_data.txt")

#done


so the first few lines for just grabbing the header and saving it separately is not needed
It must be fairly "portable" - hopefully enough for Android?

I will have a go at downloading Python for Android and see if the code will run on the file saved from the data grab using the terminal app that seems to work flawlessly with bluetooth
I will report back either way :D

I'm also sure the middle section could be merged so that the header is ignored and commas replaced by tabs in one operation and a single file write- no doubt a very easy task for a programmer who has seen Python before

My comfort zone is hammers and spanners :D

Regards
Rob
robocog
Posts: 64
Joined: Tue Sep 12, 2017 10:06 am

Re: New firmware and serial data capture to visualize and pl

Post by robocog »

Got python installed and generated a text file that should be compatible with gnuplot, just going through the different versions of gnuplot on the app store - hopefully one will work...
Will report back ;)
robocog
Posts: 64
Joined: Tue Sep 12, 2017 10:06 am

Re: New firmware and serial data capture to visualize and pl

Post by robocog »

Got it working and produced my first plot 100% on Android

Not Ideal as it took a LOT of faffing with moving files around and renaming them to produce the graph
(and seems the only option on this free version is to screen grab the image)

I can perhaps tweak the Python script to grab the newest terminal output file (name is created with a date - which makes the process a little harder than if it were to have a static name) and the directory where seemingly this version of gnuplot will only read data from is inaccessible from any file managers as far as I can tell

But - here it is - the first graph
DSO shell > Bluetooth > phone > Terminal session > Python script to tidy terminal grab > Gnuplot > screenshot > the cloud > my pc (once it synced up) > here!
Screenshot_2018-01-15-21-17-18.png
Note the power level of my mobile! - I think I started with 100%...lots of fluffing and faffing to make this work!
Next time will be quicker

Regards
Rob
RobNL
Posts: 122
Joined: Mon Dec 25, 2017 3:11 pm
Location: The Netherlands
Contact:

Re: New firmware and serial data capture to visualize and pl

Post by RobNL »

You are really good !
I take my hat off for you.
CU, Rob
robocog
Posts: 64
Joined: Tue Sep 12, 2017 10:06 am

Re: New firmware and serial data capture to visualize and pl

Post by robocog »

Whilst it is still fresh in my mind as it seems very complex
The Terminal app I used -BlueTerm

hit the 3 dots bottom right
Start logging
connect device
hit the buttons on the DSO shell to spit the data out
stop logging
close Blueterm

file is located /storage/emulated/0/blueTerm_date_time.log
this will need renaming to "alldata.txt" and moving to /storage/emulated/0/qpython (which won't exist till you have installed qpython....see next step)

Python for android
I installed QPython
The script
Screenshot_2018-01-15-21-36-38.png
This (when run) then produces a file called gnuplot.txt in /storage/emulated/0/qpython
this will need moving and renaming (see next stage!)

I installed MobileGnuplotViewerFree
the script is this - save it in the example scripts directory using the apps explorer
Screenshot_2018-01-15-21-44-18.png
You will need to move /storage/emulated/0/qpython/gnuplot.txt to the same directory as the example scripts for MobileGnuplotFree and rename it to dsodata.txt (my normal file browser cannot locate this directory so unable to make the script do it as if by magic)
Hit run and hopefully a graph will appear...
At this point there is nothing more can be done with it other than taking a screen grab
(though I think there is a paid version that will likely do a lot more)

Regards
Rob
Last edited by robocog on Mon Jan 15, 2018 10:10 pm, edited 1 time in total.
robocog
Posts: 64
Joined: Tue Sep 12, 2017 10:06 am

Re: New firmware and serial data capture to visualize and pl

Post by robocog »

If anyone has a less convoluted method of this I'm all ears as I'll never remember the whole process again, let alone under pressure, under a dashboard and away from wifi and 4g :D :D

Regards
Rob
jye1
Posts: 1221
Joined: Wed Feb 20, 2013 4:58 pm

Re: New firmware and serial data capture to visualize and pl

Post by jye1 »

Thank you guys. What you are talking about here is very interesting. I don't have time to take a close look yet. Will come back later. As to the suggestion to remove data header and change deliminator I'm thinking we might be better to make them optional and changeable by user to maximize flexibility.
RobNL
Posts: 122
Joined: Mon Dec 25, 2017 3:11 pm
Location: The Netherlands
Contact:

Re: New firmware and serial data capture to visualize and pl

Post by RobNL »

YES ! Rob deserves to be helped a bit by you, he has already moved mountains :D
CU, Rob
RobNL
Posts: 122
Joined: Mon Dec 25, 2017 3:11 pm
Location: The Netherlands
Contact:

Re: New firmware and serial data capture to visualize and pl

Post by RobNL »

@Robocog, have you see this website ?
https://developer.android.com/studio/index.html
CU, Rob
robocog
Posts: 64
Joined: Tue Sep 12, 2017 10:06 am

Re: New firmware and serial data capture to visualize and pl

Post by robocog »

I made big progress last night but too early in the morning to post. (I have to sleep before work!)
Will report back tonight after work
JYE please don't change the data!(it will break my code...ROFL)

Regards
Rob
Post Reply