Log file parser for troubleshooting

No Judgement
1

Hi Guys,

I’ve been thinking for the last few days.

It seems that we always ask for the log file whenever people run into issues that we try to solve.

Would it make sense to create a parser for the log file? It could implement several features for analyzing the log file. I would implement it as a online tool for people to upload their file to and get feedback on.

Brainstorm thoughts:

  1. Output a gcode file from the log (Would that even be possible?)
  2. Warn about decimal resolution being to “high”
  3. Output where the “buffer” keyword is found, including the previous / next 5 lines for each hit.

Let me hear what you think about that. I would be able to create a proto type of the this to show of, before we spend to much time on it.

3 Likes
2

This would be very helpful, imo.

2 Likes
3

It’s offtopic a bit, sorry but somehow related.

Why do we need the constant out flow of:
[PE:0.09,0.19,64]
<Idle,MPos:33.51,2.86,-13.00,WPos:0.000,0.000,0.000>
?
It’s kind of annoying testing grbl compatibly and checking the logs, plus creating a huge log file.

4

Totally relevant!

We either need a simpler log file, only containing the information we need to troubleshoot or we need some tools to help us doing the initial analysis.

What about 2 log files?

5

Theoretical via script, yes.
Challenges: To catch “sending” as well as “ending” with the missing ‘S’ and the lines with gcode that don’t have a sending or ending in front. From a linux perspective all tools needed are installed by default.

Edit: sure this can be done via batch on window as well

6

For windows users I believe that a online tool where you can drag and drop your file would be a major help.

7

I would throw the ball to the programmers side first, to have a more readable log file. Online is always a :persevere: for me. Maybe it’s because I’m old. :rofl:

1 Like
8

Do we have linux geeks here? I suspect that this task could be a one line terminal command with awk or sed.

9

grep look for all lines not having [ or < or ok at the start

egrep -v ‘([[<].*|ok)’ log.txt

:wink: Note - those single quotes have been changed by the forum software to “pretty quotes”. Use the standard single quote on the command line…

To watch in “real time”, open a command line in the GroundControl folder and use

tail -f log.txt | egrep -v ‘([[<].*|ok)’

(again, don’t use “pretty quotes”)

1 Like
10

Wow!

blurfl
blurfl1467×471 139 KB

now only get rid of the sending :slight_smile:
Edit: Thanks for the unexpected education. Stored in the knowledge base.

1 Like
11

Try

tail -f log.txt | egrep -v ‘([[<].*|ok|sent)’

On mobile and not Linux user. Just guessing :grinning:

1 Like
12

Looks like we you guys have created a log-to gcode!
log-to-gcode

2 Likes
13

All credits should go to to the master mind: @blurfl :smile:

@Gero - could you run the output from the log in a gcode simulator? That was actually what I was trying to achieve with all my talk about the parser :wink:

2 Likes
14

Still had to remove a lot of text like ‘turning spindle of or on’, ‘not supported and ignored’ and other stuff.
The buffer overflow needed to be removed. Looks good from a quick view, but has issues. No time left today.
loggcode.nc (127.4 KB)
logtogcode

2 Likes
15

We either need a simpler log file, only containing the information we need to
troubleshoot or we need some tools to help us doing the initial analysis.

we don’t know what info we need in the file until we have a description of the
problem

What about 2 log files?

what do you put in one rather than the other?

issues tend to come is bursts, we start having people run into a problem, we
diagnose it and implement a fix, and that problem stops showing up as much.

16

raises hand.

yes, this sort of thing could be done trivially, but the issue is defining what
needs to be done

grep -e ".[0-9][0-9][0-9][0-9] file
will show any lines that have four or more digits past the decimal

grep -e buffer -A 5 -B 5 file
will show any lines that have buffer in them, and 5 lines before and after.

2 Likes
17

I thought this thread was really helpful. One more thing to add is that the regular expression:

\[(.*?)\] will match anything between the [] characters…mostly saying it so that I can look this up again later :wink:

1 Like
18

O’Reilly has a good book, “Mastering Regular Expressions”.

3 Likes