M4: Cut fail. not much info, but potential Z axis bug after stop

Was cutting the Maslow logo. It got to the starting few cuts of the bottom later, then Fail. Here is what happened.

  1. I got a call and my phone’s browser disconnected somehow. So I closed the browser and it continued to cut so I was just watching and waiting for it to finish the cut and after a minute or so, it just stopped moving. When I reconnected, I don’t see much in the error log except that Emergency stop triggered over and over as expected really.

Feature request: (maybe I can help do this… not sure) I wish there was a log stored on the device so we could see these errors after the fact…I think that would be super useful.

Now to the bug: After it had the emergency stop and I turned off the router and vacuum, z was going up and down:

I’m pretty sure it should not move anything once its in this state so this seems like something to pursue.

I’ll regroup and try some more cuts after I get a better solution for what to have out by the device that I can keep connected.

Here is how close I was:

2 Likes

I took a quick glance and fluidnc code and I don’t know if it would be a good idea to stream the serial output to an SD file, as I’m not sure how that would impact the performance of the device. Maybe a log buffer that could be stored periodically would be better? Anyway, I"m thinking about other options.

1 Like

So today I went and got an old laptop, parked it out in the garage, which is a bit chilly this morning (55F/12C) and started it back up, went back to home, reset Z home, then ran the maslow.nc file again. It got further (down to last layer!) then I got:

[MSG:WARN: Encoder read failure on 2]
[MSG:ERR: PANIC! Stopping all motors]
[MSG:WARN: Bad connection on Top Left encoder, failed to read 1 times in the last second]
[MSG:ERR: PANIC! Stopping all motors]
...

I’m assuming this could be an issue, lol. I did notice that it kept sending g-code after the stop. I’m sure it did last time too, but I didn’t have an active serial connection so did not see what it was complaining about.

Other than bad hardware, what might be causing this?

1 Like

That sounds like hardware to me. Unplugging and repluging the top left ethernet cable is the place to start.

Do you get that message all the time or just that once?

If you press test does everything look normal?

Just that one and one a few messages later. After restart it was fine and test looks ok. I suspect the initial fail on this topic too but i did not see the messages because of the disconnect. I’m concerned given two failures in a row. Ill try again when i can and see if it does it again

1 Like

I did try again (0.70.2) got even further this time, then same error. (log attached)

This time when it failed, the machine moved the wrong direction on me, but I turned off the router. Then I noted again that even though “e-stopped” the gcode was still going and Z axis was moving up and down periodically (may have been the g-code which was still running, and it was on the layer doing tabs so that would make sense)

After pulling the plug fully and plugging it back in, I was able to release, retract all and the encoder was fine seemingly. I suspect something is wrong either with the encoder or main board? loose connection? I am probably not equipped enough to diagnose it but if you have any suggestions I can try I’m game to try.

Current “test” reports:

[GC:G0 G54 G17 G21 G90 G94 M5 M9 T0 F0 S0]
Index.html Version: 0.7
[MSG:INFO: Firmware Version: 0.70]
[MSG:INFO: Motor detected on Top Left]
[MSG:INFO: Encoder connected on Top Left]
[MSG:INFO: Magnet detected on Top Left]
[MSG:INFO: Motor detected on Top Right]
[MSG:INFO: Encoder connected on Top Right]
[MSG:INFO: Magnet detected on Top Right]
[MSG:INFO: Motor detected on Bottom Left]
[MSG:INFO: Encoder connected on Bottom Left]
[MSG:INFO: Magnet detected on Bottom Left]
[MSG:INFO: Motor detected on Bottom Right]
[MSG:INFO: Encoder connected on Bottom Right]
[MSG:INFO: Magnet detected on Bottom Right]

cut-m-log-04-29.txt (26.4 KB)

oh, and here’s the carnage (no, not the junky mdf I’m using :smile: … the bottom right of the logo ):

@bar, @RomanG

First, C++ is not my best language. I’d say “novice” at best.

To try to address g-code (and z steppers) running after stop I saw on my encoder error, I am looking through fluidnc code, and in looking through Maslow.cpp, To stop running gcode and steppers, should we pull in “Protocol.h” and call protocol_send_event_from_ISR(&cycleStopEvent); on eStop and panic after setting alarm state? Or is there another place these things happen naturally?

2 Likes

I think that you have totally identified the issue (the gcode continues to run and move the zaxis after the estop stops the four servos) and I think that could be a good fix!

I wonder if we also want to set sys.state to Alarm to prevent the user from clicking the jog buttons? Setting sys.state to alarm might even end the cycle automatically?

Maslow.cpp already sets sys.state to alarm in these, but does not call the stuff that stops running gcode or steppers, so it appears that this will not eventually stop things… but I have a hard time following the states and loop(s) going on in this environment. I know that when the fluidnc generic code has a stop it calls an internal method in protocol that the above also calls to stop all the things… or at least that is what I see…

1 Like

Makes sense to me! I think that I thought that setting sys.state = Alarm would do that automatically but clearly that’s not the case :stuck_out_tongue:

Do you feel good about making a pull request for that change?

Sure, let me build and run it on my device and hit stop and test it out. then I’ll do a pr. (it may be tomorrow, I’ll see if I can get the time to do it today after work or not)

4 Likes

Turns out that did not work, but I found the magic functions that do it better. I’ll also include a snippet to just print the error message once so the log is better. I’ll get a PR together once I figure out how to test panic. maybe pull out an ethernet cable while its running?

3 Likes

PR has been put out there…

@bar I’m by no means good at electronics, but I do have a basic meter and I’m willing to try to diagnose my encoder errors if you can think of anything I can do to diagnose it.

1 Like

You are still getting them? I thought that maybe that was a one time thing.

Can you give me a summary about what you are seeing now and what the pattern seems to be?

Twice, maybe 3 times. (Once no logs. ) I got the top left encoder errors. Logs above, but it was some time into the cut and all was fine before. and after restarting nothing was unusual.

1 Like

And it’s always top left?

This sounds crazy but maybe it’s related to the push pull issue which seems to happen more often on the top left too

yes, top left both times I had logs. I don’t see anything unusual in belt tension or anything, but I guess that could be possible. I will re-assemble with another arm there just to check that its not something there (although if its the board then it would not be useful)

1 Like

I’m going to dig into the push pull issue on the TL encoder today and maybe I’ll learn something on the software side

I tried on the latest Maslow-Main (0.71) and got through the maslow logo over this used up scrap of mdf with no top left encoder errors! (the gouge is from one of the failures).

I am a bit hesitant to try anything bigger right now with the bit, but I’m going to run through a “dry” larger job and see if it will behave now… That is, after I get pause / stop / resume working again (resume does not work on current ui, and my stop changes work but no way to resume from it since the pause button never changes to resume)

3 Likes