I appreciate all the thoughts you have put into this. I can follow most of it.
Throwing out another idea. Think of when installing software on a PC. There is usually a standard setup (ie easier mode, less prompts) or an advanced/define options mode.
That might be a way also to let the user decide if they want to risk choosing manual definitions.
Dano
I’ve just about surfaced after a 3 days of demoing all of the boardgames 
Yeah, I tend to agree, though my take (and @bar likely will disagree) is the standard setup should still be asking for info / measurements from the user, they should just be hand-holding the user through it all like a Setup Wizard, and sanity checking / validating them - with the advanced being documented but letting the user do what they want.
Here’s another simple to do one from looking at a thread this morning:
I don’t think the logs have the values from the Maslow.yaml (until it sets some at the end)?
I’d dump at least the user-modifiable ones (Z offsets, Vert/Horizontal - actually i’ll just put below) both immediately when the user hits the Calibrate Button, and after you’ve written them at the end of calibration (input/output logging basically).
My experience is users i’ve had are about 10x more likely to mangage/bother to capture 1 log 1 way, then 2 logs 2 ways.
(I’d just have this / these fields dumped in the log:
Maslow_vertical: false
Maslow_calibration_grid_width_mm_X: 1200.000000
Maslow_calibration_grid_height_mm_Y: 800.000000
Maslow_calibration_grid_size: 7
Maslow_tlX: -9.100000
Maslow_tlY: 2276.100098
Maslow_trX: 2910.899902
Maslow_trY: 2277.899902
Maslow_blX: 0.000000
Maslow_blY: 0.000000
Maslow_brX: 2926.600098
Maslow_brY: 0.000000
Maslow_tlZ: 90.000000
Maslow_trZ: 50.000000
Maslow_blZ: 30.000000
Maslow_brZ: 68.000000
Maslow_Retract_Current_Threshold: 900
Maslow_Calibration_Current_Threshold: 900
Maslow_Acceptable_Calibration_Threshold: 0.450000
Maslow_Extend_Dist: 1700.000000
Maslow_beltEndExtension: 30.000000
Maslow_armLength: 123.400002
Maslow_Scale_X: 1.000000
Maslow_Scale_Y: 1.000000
)
Not much progress on this, got distracted by:
Everything’s better under the C (a quick poke at the frame flex value).
But I did try a quick manual iteration of doing short descents (100) multiple times, weighted average of the results, then using that as the input for another round, with the area the initial sample is taken from decreasing each time.
I started with a roughly-right frame measurement (in unstretched-belt-mm), and only did 10 samples-with-100-descents in each step as this seemed like the lowest I was likely to get anything from. And I did 3 iterations, sample ± 50mm in the first, then ±25mm, then ±12mm.
| Fitness | TL X | TL Y | TR X | TR Y | BL X | BL Y | BR X | BR Y |
|---|---|---|---|---|---|---|---|---|
| 1st round | 100 | |||||||
| plus minus 50mm | ||||||||
| 0.5715 | -10.3 | 2314 | 2868.6 | 2327.9 | 0 | 0 | 2890.1 | 0 |
| 0.5041 | 3.9 | 2330.4 | 2875.4 | 2320.8 | 0 | 0 | 2890.9 | 0 |
| 0.4197 | 7.8 | 2319.8 | 2899 | 2291.6 | 0 | 0 | 2903.3 | 0 |
| 0.3712 | 7.6 | 2293.5 | 2924.9 | 2258.8 | 0 | 0 | 2924.6 | 0 |
| 0.6179 | -16 | 2298.2 | 2878.9 | 2315.7 | 0 | 0 | 2897.2 | 0 |
| 0.4399 | -10.7 | 2264.1 | 2898.2 | 2292.5 | 0 | 0 | 2929.9 | 0 |
| 0.2687 | 14 | 2307.9 | 2924.8 | 2258.9 | 0 | 0 | 2918.8 | 0 |
| 0.6162 | -10.9 | 2286.6 | 2887.6 | 2305.1 | 0 | 0 | 2911.5 | 0 |
| 0.6118 | -0.8 | 2324.3 | 2876.3 | 2319.3 | 0 | 0 | 2891.3 | 0 |
| 0.9893 | -7.2 | 2273.6 | 2910.8 | 2276.3 | 0 | 0 | 2925.8 | 0 |
| Weight total | ||||||||
| 5.4103 | ||||||||
| Weighted | ||||||||
| -5.88645 | 1322.451 | 1639.4049 | 1330.39485 | 0 | 0 | 1651.69215 | 0 | |
| 1.96599 | 1174.75464 | 1449.48914 | 1169.91528 | 0 | 0 | 1457.30269 | 0 | |
| 3.27366 | 973.62006 | 1216.7103 | 961.78452 | 0 | 0 | 1218.51501 | 0 | |
| 2.82112 | 851.3472 | 1085.72288 | 838.46656 | 0 | 0 | 1085.61152 | 0 | |
| -9.8864 | 1420.05778 | 1778.87231 | 1430.87103 | 0 | 0 | 1790.17988 | 0 | |
| -4.70693 | 995.97759 | 1274.91818 | 1008.47075 | 0 | 0 | 1288.86301 | 0 | |
| 3.7618 | 620.13273 | 785.89376 | 606.96643 | 0 | 0 | 784.28156 | 0 | |
| -6.71658 | 1409.00292 | 1779.33912 | 1420.40262 | 0 | 0 | 1794.0663 | 0 | |
| -0.48944 | 1422.00674 | 1759.72034 | 1418.94774 | 0 | 0 | 1768.89734 | 0 | |
| -7.12296 | 2249.27248 | 2879.65444 | 2251.94359 | 0 | 0 | 2894.49394 | 0 | |
| Weighted Totals | ||||||||
| -22.98619 | 12438.62314 | 15649.72537 | 12438.16337 | 0 | 0 | 15733.9034 | 0 | |
| Weighted Averages | ||||||||
| -4.248598044 | 2299.063479 | 2892.579962 | 2298.978498 | 0 | 0 | 2908.138809 | 0 | |
| 2nd round | 100 | |||||||
| plus minus 25mm | ||||||||
| 0.8812 | 12 | 2308.4 | 2881.8 | 2280 | 0 | 0 | 2932.3 | 0 |
| 0.8812 | -1.6 | 2304.6 | 2868.4 | 2314.6 | 0 | 0 | 2930.1 | 0 |
| 0.8812 | -11 | 2308.7 | 2898.3 | 2308.1 | 0 | 0 | 2927.9 | 0 |
| 0.8812 | 14.7 | 2324 | 2869.7 | 2273.5 | 0 | 0 | 2931.8 | 0 |
| 0.9488 | -5.5 | 2285.4 | 2909.1 | 2278.4 | 0 | 0 | 2918 | 0 |
| 1.0708 | -7.2 | 2294.5 | 2895.3 | 2295.6 | 0 | 0 | 2909.1 | 0 |
| 0.8877 | -4.3 | 2308.1 | 2888.3 | 2304.3 | 0 | 0 | 2901 | 0 |
| 0.8812 | 5.7 | 2311.3 | 2915.8 | 2277.1 | 0 | 0 | 2908.8 | 0 |
| 0.8812 | -5.2 | 2276.4 | 2908 | 2307.9 | 0 | 0 | 2904.9 | 0 |
| 0.9302 | -4.6 | 2292 | 2904.8 | 2283.8 | 0 | 0 | 2913.6 | 0 |
| Weight total | ||||||||
| 9.1247 | ||||||||
| Weighted | ||||||||
| 10.5744 | 2034.16208 | 2539.44216 | 2009.136 | 0 | 0 | 2583.94276 | 0 | |
| -1.40992 | 2030.81352 | 2527.63408 | 2039.62552 | 0 | 0 | 2582.00412 | 0 | |
| -9.6932 | 2034.42644 | 2553.98196 | 2033.89772 | 0 | 0 | 2580.06548 | 0 | |
| 12.95364 | 2047.9088 | 2528.77964 | 2003.4082 | 0 | 0 | 2583.50216 | 0 | |
| -5.2184 | 2168.38752 | 2760.15408 | 2161.74592 | 0 | 0 | 2768.5984 | 0 | |
| -7.70976 | 2456.9506 | 3100.28724 | 2458.12848 | 0 | 0 | 3115.06428 | 0 | |
| -3.81711 | 2048.90037 | 2563.94391 | 2045.52711 | 0 | 0 | 2575.2177 | 0 | |
| 5.02284 | 2036.71756 | 2569.40296 | 2006.58052 | 0 | 0 | 2563.23456 | 0 | |
| -4.58224 | 2005.96368 | 2562.5296 | 2033.72148 | 0 | 0 | 2559.79788 | 0 | |
| -4.27892 | 2132.0184 | 2702.04496 | 2124.39076 | 0 | 0 | 2710.23072 | 0 | |
| Weighted Totals | ||||||||
| -8.15867 | 20996.24897 | 26408.20059 | 20916.16171 | 0 | 0 | 26621.65806 | 0 | |
| Weighted Averages | ||||||||
| -0.894130218 | 2301.034442 | 2894.14453 | 2292.257467 | 0 | 0 | 2917.537898 | 0 | |
| 3rd round | 100 | |||||||
| plus minus 12mm | ||||||||
| 1.0259 | -4.5 | 2291.4 | 2902.7 | 2286.6 | 0 | 0 | 2914.4 | 0 |
| 0.9889 | -4.5 | 2291.9 | 2903.3 | 2285.6 | 0 | 0 | 2913.8 | 0 |
| 0.9364 | -3.4 | 2300.3 | 2897.2 | 2293.3 | 0 | 0 | 2908.2 | 0 |
| 0.8446 | -1.8 | 2301.4 | 2898.4 | 2291.9 | 0 | 0 | 2908.8 | 0 |
| 0.8978 | -5.6 | 2307.4 | 2888.2 | 2304.1 | 0 | 0 | 2900.2 | 0 |
| 1.003 | -6.4 | 2300.7 | 2891.3 | 2300.5 | 0 | 0 | 2904.7 | 0 |
| 0.8056 | -1.1 | 2301.8 | 2899 | 2291.1 | 0 | 0 | 2909.1 | 0 |
| 1.0501 | -5 | 2293.5 | 2900.2 | 2289.4 | 0 | 0 | 2912.1 | 0 |
| 0.8971 | -3.5 | 2306.8 | 2890.2 | 2302 | 0 | 0 | 2902.9 | 0 |
| 1.0043 | -5.3 | 2287.3 | 2906.4 | 2281.8 | 0 | 0 | 2916.6 | 0 |
| Weight total | ||||||||
| 9.4537 | ||||||||
| Weighted | ||||||||
| -4.61655 | 2350.74726 | 2977.87993 | 2345.82294 | 0 | 0 | 2989.88296 | 0 | |
| -4.45005 | 2266.45991 | 2871.07337 | 2260.22984 | 0 | 0 | 2881.45682 | 0 | |
| -3.18376 | 2154.00092 | 2712.93808 | 2147.44612 | 0 | 0 | 2723.23848 | 0 | |
| -1.52028 | 1943.76244 | 2447.98864 | 1935.73874 | 0 | 0 | 2456.77248 | 0 | |
| -5.02768 | 2071.58372 | 2593.02596 | 2068.62098 | 0 | 0 | 2603.79956 | 0 | |
| -6.4192 | 2307.6021 | 2899.9739 | 2307.4015 | 0 | 0 | 2913.4141 | 0 | |
| -0.88616 | 1854.33008 | 2335.4344 | 1845.71016 | 0 | 0 | 2343.57096 | 0 | |
| -5.2505 | 2408.40435 | 3045.50002 | 2404.09894 | 0 | 0 | 3057.99621 | 0 | |
| -3.13985 | 2069.43028 | 2592.79842 | 2065.1242 | 0 | 0 | 2604.19159 | 0 | |
| -5.32279 | 2297.13539 | 2918.89752 | 2291.61174 | 0 | 0 | 2929.14138 | 0 | |
| Weighted Totals | ||||||||
| -39.81682 | 21723.45645 | 27395.51024 | 21671.80516 | 0 | 0 | 27503.46454 | 0 | |
| Weighted Averages | ||||||||
| -4.211771053 | 2297.878762 | 2897.86118 | 2292.415156 | 0 | 0 | 2909.280445 | 0 |
The output of the 3rd round (equiv to 3000 descents in the normal algorithm) Is pretty good, the TLX is slightly off - looking at the data it looks like you get a spread of potential values, possibly bimodal, which is interesting. I have wondered if doing a grid of points is giving us sampling artifacts and we should be doing (still stratified) random sample points for the calibration, that’s for another day though.
That IS interesting, I am curious to see if that pattern holds up as something which consistently happens
Throwing this message in here because I like the research that @RandomDave is doing, but I have a profoundly different idea for how to perform the actual calibration, which would feed into different ways to establish frame flex / belt stretch as well.
I’ve had a poke around inside the FluidNC code for this, but I wanted to get pointers (probably from @bar ) as to where in the code I should be looking to do what I need to.
I can plainly see the code to move one belt at a time. My need, however, is to retract / extend two belts simultaneously, while the other two don’t move (or at most spool out a little bit).
Check out the the function for taking a measurement. That’s basically exact what it does, two belts are held steady and the other two retract. (It wouldn’t be too hard to modify for different behavior)
Lots of different approaches are good - there’s a decent chance what i’m trying won’t come to anything, and you never know what crossovers will give us what we want too 
What you want to do i’d definitely be interested in doing too!
@bar or @md8n (or anyone else) - just a sanity check.
The point data output to serial, like:
CLBM:[{bl:1844.02, br:1863.70, tr:1853.19, tl:1853.05},{bl:1963.27, br:1742.44, tr:1738.79, and so on.
Has that had any projection done on it already (via the ZOffsets or anything else) or is it literally the belt length via the magnet sensor converted to mm?
(this is on recent FW - 1.06 though I doubt it’s changed recently)
These values have already been projected into the XY plane accounting for the height of the z-axis.
Ahhhhh, that might account for some funkiness i’m seeing with a specific calibration run, thanks!
I’ve been playing with a bit of classic Monte Carlo stuff (technically I think it would count as MCMC, but with a binary yes/no rather than a weighted step), that seems to be working well in my small experiments.
If we’re in the right ballpark for the answer, just sampling around what we’ve got and keeping any higher fitness gets me better fitness than I was seeing from the algorithm.
We can use a descending radius to sample from too to focus in those samples once we’re close.
That said, we need to be careful we don’t fall in the same trap of getting stuck on a non-optimal minima. And we need to get fairly close to a minima to start. What seems to work quite well is to marry a short run of the original algorithm (up to 1000 steps) with some MC, and loop that, keeping the current best but kicking off new descents for a decent distance away.
It’s scrappy experiment code, but this is working quite well for me:
const compute1kButton = document.getElementById('compute-1k-button');
compute1kButton.addEventListener('click', () => {
currentBest = JSON.parse(JSON.stringify(initialGuess));
for(let outer = 0; outer < 10; outer++){
clearCanvas();
guessThisTime = JSON.parse(JSON.stringify(currentBest));
guessThisTime.tl.x = guessThisTime.tl.x + (Math.random() * 10) - 5;
guessThisTime.tl.y = guessThisTime.tl.y + (Math.random() * 10) - 5;
guessThisTime.tr.x = guessThisTime.tr.x + (Math.random() * 10) - 5;
guessThisTime.tr.y = guessThisTime.tr.y + (Math.random() * 10) - 5;
guessThisTime.br.x = guessThisTime.br.x + (Math.random() * 10) - 5;
guessThisTime = computeLinesFitness(measurements, guessThisTime);
guessThisTime = findMaxFitness(guessThisTime, measurements);
printResults(guessThisTime);
if(1/guessThisTime.fitness > 1/currentBest.fitness){
console.log("Fitness: " + 1/guessThisTime.fitness);
currentBest = JSON.parse(JSON.stringify(guessThisTime));
}
for(let i = 0; i < 1000; i++){
let iterations = 10;
for(let focuser = 0; focuser < iterations; focuser++){
let range = 10 * (focuser / iterations);
let halfRange = range/2;
clearCanvas();
refineThisTime = JSON.parse(JSON.stringify(guessThisTime));
if(Math.random() < 0.3){ refineThisTime.tl.x = refineThisTime.tl.x + (Math.random() * range) - halfRange; }
if(Math.random() < 0.3){ refineThisTime.tl.y = refineThisTime.tl.y + (Math.random() * range) - halfRange; }
if(Math.random() < 0.3){ refineThisTime.tr.x = refineThisTime.tr.x + (Math.random() * range) - halfRange; }
if(Math.random() < 0.3){ refineThisTime.tr.y = refineThisTime.tr.y + (Math.random() * range) - halfRange; }
if(Math.random() < 0.3){ refineThisTime.br.x = refineThisTime.br.x + (Math.random() * range) - halfRange; }
refineThisTime = computeLinesFitness(measurements, refineThisTime);
if(1/refineThisTime.fitness > 1/guessThisTime.fitness){
console.log("Fitness: " + 1/refineThisTime.fitness);
guessThisTime = JSON.parse(JSON.stringify(refineThisTime));
}
if(1/guessThisTime.fitness > 1/currentBest.fitness){
console.log("Fitness: " + 1/guessThisTime.fitness);
currentBest = JSON.parse(JSON.stringify(guessThisTime));
}
}
}
printResults(currentBest);
}
// printResults(initialGuess);
printResults(currentBest);
});
(theres a few other mods to the code to make this work too)
one of the things to note is:
if(Math.random() < 0.3){ refineThisTime.tl.x = refineThisTime.tl.x + (Math.random() * range) - halfRange; }
if(Math.random() < 0.3){ refineThisTime.tl.y = refineThisTime.tl.y + (Math.random() * range) - halfRange; }
if(Math.random() < 0.3){ refineThisTime.tr.x = refineThisTime.tr.x + (Math.random() * range) - halfRange; }
if(Math.random() < 0.3){ refineThisTime.tr.y = refineThisTime.tr.y + (Math.random() * range) - halfRange; }
if(Math.random() < 0.3){ refineThisTime.br.x = refineThisTime.br.x + (Math.random() * range) - halfRange; }
It’s important that we’re potentially randomising only some of the values - for resampling, we want to try small variations on the current as well as medium ones. 0.3 is arbitrary but I think ok - it’s something that could do with testing with a proper data set.
When I find some time i’ll tidy it up into some commits other people can try. I suspect it might well work best with a round of weighted mean as an early way of getting ballpark measurements to save a bit of time, and emphasising a bit more robustness 
I love this approach, that seems like a very promising avenue of research.
I think that the real gold standard is if we run it 10x in a row on the same frame how close to the same locations do we get? The perfect system should give us the same numbers every time.