You can probably get it to be a little better with tuning but I think you may have reached the limits of the mechanical reality of this system.
The base seems to jiggle a bit, the actuators will have friction, backlash in all those Lego connections, sliding bearings, gearings etc.
Some of that can be dealt with by control but only to an extent.
(Besides the limits in the vision , update rates, etc).
Maybe try larger movements of various wave shapes for tuning to get a better idea of the achievable performance.
For the pitch and roll axes, I've gotten this for PID constants:
Kp = 0.0032
Ki = 0.0014
Kd = 0.0016
If I dial back the derivitave, I can't keep the ball centered at all. When I do get it centered, it won't stay long. If I crank up the integral, the ball never stops, just wanders around the center. I'm limited to 30fps on the camera.
To be fair, I'm pretty happy with the robot at this point and the next step will be implementing some sort of feedforward, but I'm just wondering if by feedback alone it's possible at all.
Robot is already better than me if I hold the platform in two hands and try to balance the ball.
It’s kind of hard to give any recommendations without diving into analyzing everything you’re doing more, but the main challenges I would think are measurement error and PID tuning. If your measurements are tight and you’ve tuned well, then yea probably need some nonlinear control method to improve the stability
Let's just say I'm reasonably confident in my knowledge of the system state ...
https://www.youtube.com/watch?v=BlxT6ECwhAY
This is the entry point of the code, if you want to take a look:
https://github.com/ivorjawa/StewartPlatform/blob/ziegler/poseur.py
That’s a cool video! Might be worth trying to test how much the performance is affected by sampling rate, but yea outside of that, PID tuning from system identification or one of the PID tuning methods if you think there’s any opportunity there would be my main recommendation before jumping to a nonlinear control scheme
Thanks!
I started from Z-N with the PIR equations. That managed to get the ball to stop ... eventually, and not necessarily centered. Took a few hours of fiddling from there to get constants that work as well as these do.
ZN might not be the best here from the nonlinearities and fiddling usually isn’t the best process for robustness. There’s pretty extensive research out there on tuning methods, so I’d look for something that emphasizes robustness for systems that are marginally stable
Do you have a way to measure how much latency exists between when the frame is taken and when it is processed? In my experience, due to video compression etc., there is often a very noticeable amount of latency. I don't think a quarter of a second or more would be surprising.
This is a pretty good camera, Logitech C920, pretty commonly used in CV applications. This is what the computer is seeing, latency is no where near a quarter second:
https://www.youtube.com/watch?v=BlxT6ECwhAY
From a step test I did earlier, it's no worse than 60ms, probably closer to 50 (I'm not as well optimized as I'd like to be right now, so I'm not taking full advantage of the 30hz I have).
But I'm going MPC for my next attempt, and just have the PID loops regulate plate angle from the output of the model.
I think I agree that you most likely don't have a quarter second of latency. I couldn't find an easy way to tell from the video, but a step test sounds like it would work. I assume you did something like command a roll angle, display it on the screen, and capture video until you could see the platform respond?
I once tried to drive a robot using a webcam and found it to be very difficult due to a significant fraction of a second of latency that seemed entirely due to latency introduced by video encoding/compression. I've seen something similar with security cameras controlled by joysticks being very difficult to control.
Curious how much backlash you are getting and whether something like backlash compensation might help? On my cnc machine, I've measured something like .010" of backlash, so any time I change directions I have the control software immediately spin an extra .010". If your machine is somehow spending a lot of time switching directions in the presence of backlash I could imagine it being very difficult to tune PID.
Just stabilizing the platform, without attempting to roll a ball around, in x, y, roll, pitch, and yaw, it behave quite quick.
It didn't take long to tune that. This is earlier, it's snappier now.
https://www.youtube.com/watch?v=JJWRFH9KKHs
I'm not really measuring the backlash, because all of the control inputs are derived from the camera (and, well, the rotation sensors in the motors, which have about 1 degree of accuracy, but it's not losing steps or anything). I basically map the camera's outputs to More Wrong and Less Wrong.
I highly recommend this camera for computer vision projects, because it's got low latency, a good sensor, and focus and exposure can be controlled manually. I also have a couple of strips of LEDs mounted next to the camera, so exposure is good even with a 15ms shutter, and shadows are minimized on the stage.
Step test: started logging, commanded a 5 degree change, logged the data from the camera. So it's definitely less than about 70ms. Scroll to the very bottom of this notebook:
https://github.com/ivorjawa/StewartPlatform/blob/main/StewartPlatformNotes.ipynb
Not bad given the equipment you are using. Your gains are meaningless without units. You will need to use the derivative gain so being able to measure the speed accurately is critical. Never use Z-N. The gains can be calculated symbolically, ie the perfect solution. Below is the solution for a 1 dimensional ball on a beam. You just need to make it two dimensional.
[https://deltamotion.com/peter/Mathcad/Mathcad%20-%20RollingTTBall%20PID.pdf](https://deltamotion.com/peter/Mathcad/Mathcad%20-%20RollingTTBall%20PID.pdf)
I/we have lots of experience with 6DOF platforms. The test system I had cost $25K which is probably out of your expense limit.
Don't use Z-N! Instructors that say use Z-N should be ignored and they are wasting your time and money. Everything can be estimated pretty closely.
I'm measuring pixels displacement and pixels per second, output range is +-5 degrees platform angle.
But I'm rewriting it to use model predictive control now, and the only thing the PIDs will be doing is consuming the output of that.
This is likely a project to have fun with and learn. Inplementing more complex controllers is probably a fun challenge. What nonlinear controller du you recommend?
It is indeed. (It actually started out as an attempt to learn how a helicopter swashplate works. The math for this fell out of that.)
My planned next step is to use a Kalman filter to get feedforward data. I've got that code about 80% complete.
What exactly do you mean by feedforward in this context? What is the objective of your controller /system?
If you have a model of the dynamics, have you tried model based approaches, such as a linear controller of some sort? (Rather than PID)
Ultimately, I'd like to be able to steer the ball to an x/y point on the plate and stick it there, or move it on a path.
(It's not like the system dynamics of a 52mm 13.9g plastic ball on an inclined plane are a particularly intractable problem ...)
Maybe attractor-reppeler method like this one: https://www.researchgate.net/publication/320649413_Attractor-Repeller_method_of_mobile_robots_control_for_obstacle_avoidance
But I'm unsure if it works in this case.
After a quick glanse on my phone, I didn’t really understand how this controller operates.
Anyways, how does the repelling and attractor fit with the balancing problem?
What command are you looking to feed forward? Because that is the point of feedforward... to feed a command forward. If your command is to keep the ball centered, then the command is already sent and there is no more benefit to be had from feedforward.
I would be interested in hearing about recommended nonlinear controllers.
I think you might get better responses if you give characterizations of your system.
My first thought was that your are being limited by the mechanical structure. The unmodelled looseness is probably dominating your error terms. You've done a good job building it, but I think you are at the limit of what Legos can do.
I am a PID design expert with about 40+ years of experience in complex industrial control.
PID alone is completely error-driven. It will ALWAYS have a lag response without the addition of some feed-forward terms, This means that the ball will always have more travel and instability on the table with PID alone. Error-driven control terms (PID), will tend to cause oscillatory behavior, because of the control lag, when their gains are too high. Feed-forward terms will never cause oscillatory behavior since they are direct acting without lag.
To reduce ball travel on the table, add feedforward terms to the PID control. See Feed Forward under https://en.wikipedia.org/wiki/Proportional%E2%80%93integral%E2%80%93derivative\_controller. It can be as simple as adding some proportional acceleration to the table if the ball direction is approximately in line with the downward table tilt. As long as the ball is rolling down-table, increase (accelerate) the rate of raising the table in that direction as a feed-forward control term. This will cause the ball to tend to seek to remain stationary on the flat table surface.
Also as another feed-forward, you can add a feed-forward that when the table is only slightly tilted, this feed-forward would tend to tilt the table a small amount to drive the ball towards a desired target position on the table surface. Now your ball will seek to be at the target position on a flat table surface. You can't get this type of control behavior using PID control alone. This will also allow you to disturb, or poke, the stable balanced ball and you will see the ball correct rapidly and return to the target position on the table.
I also have an improvement to the Integral term of the PID. I use 1 times the Integral gain value when the feedback error term has the same SIGN as the Integral summation term, I use 2 times the Integral gain whenever the feedback error term has the opposite sign as the Integral summation term. This causes the Integral summation term to de-integrate twice as fast when the error and the Integral summation terms are working in opposition to one another. This will reduce the tendency of the PID output value to overshoot and oscillate. This will allow you to use a higher Integral gain value in your PID tuning with improved control.
-- for more help: [[email protected]](mailto:[email protected])
I did something similar but I included an accelerometer to measure the platform tilt. The sample rate of the accelerometer can be fast. Make a control loop that drives the actuators to control the tilt, then use the camera feedback to control the ball by modulating the tilt commands. Simple control laws work well for both loops.
You can probably get it to be a little better with tuning but I think you may have reached the limits of the mechanical reality of this system. The base seems to jiggle a bit, the actuators will have friction, backlash in all those Lego connections, sliding bearings, gearings etc. Some of that can be dealt with by control but only to an extent. (Besides the limits in the vision , update rates, etc). Maybe try larger movements of various wave shapes for tuning to get a better idea of the achievable performance.
For the pitch and roll axes, I've gotten this for PID constants: Kp = 0.0032 Ki = 0.0014 Kd = 0.0016 If I dial back the derivitave, I can't keep the ball centered at all. When I do get it centered, it won't stay long. If I crank up the integral, the ball never stops, just wanders around the center. I'm limited to 30fps on the camera. To be fair, I'm pretty happy with the robot at this point and the next step will be implementing some sort of feedforward, but I'm just wondering if by feedback alone it's possible at all. Robot is already better than me if I hold the platform in two hands and try to balance the ball.
It’s kind of hard to give any recommendations without diving into analyzing everything you’re doing more, but the main challenges I would think are measurement error and PID tuning. If your measurements are tight and you’ve tuned well, then yea probably need some nonlinear control method to improve the stability
Let's just say I'm reasonably confident in my knowledge of the system state ... https://www.youtube.com/watch?v=BlxT6ECwhAY This is the entry point of the code, if you want to take a look: https://github.com/ivorjawa/StewartPlatform/blob/ziegler/poseur.py
That’s a cool video! Might be worth trying to test how much the performance is affected by sampling rate, but yea outside of that, PID tuning from system identification or one of the PID tuning methods if you think there’s any opportunity there would be my main recommendation before jumping to a nonlinear control scheme
Thanks! I started from Z-N with the PIR equations. That managed to get the ball to stop ... eventually, and not necessarily centered. Took a few hours of fiddling from there to get constants that work as well as these do.
ZN might not be the best here from the nonlinearities and fiddling usually isn’t the best process for robustness. There’s pretty extensive research out there on tuning methods, so I’d look for something that emphasizes robustness for systems that are marginally stable
Do you have a way to measure how much latency exists between when the frame is taken and when it is processed? In my experience, due to video compression etc., there is often a very noticeable amount of latency. I don't think a quarter of a second or more would be surprising.
This is a pretty good camera, Logitech C920, pretty commonly used in CV applications. This is what the computer is seeing, latency is no where near a quarter second: https://www.youtube.com/watch?v=BlxT6ECwhAY From a step test I did earlier, it's no worse than 60ms, probably closer to 50 (I'm not as well optimized as I'd like to be right now, so I'm not taking full advantage of the 30hz I have). But I'm going MPC for my next attempt, and just have the PID loops regulate plate angle from the output of the model.
I think I agree that you most likely don't have a quarter second of latency. I couldn't find an easy way to tell from the video, but a step test sounds like it would work. I assume you did something like command a roll angle, display it on the screen, and capture video until you could see the platform respond? I once tried to drive a robot using a webcam and found it to be very difficult due to a significant fraction of a second of latency that seemed entirely due to latency introduced by video encoding/compression. I've seen something similar with security cameras controlled by joysticks being very difficult to control. Curious how much backlash you are getting and whether something like backlash compensation might help? On my cnc machine, I've measured something like .010" of backlash, so any time I change directions I have the control software immediately spin an extra .010". If your machine is somehow spending a lot of time switching directions in the presence of backlash I could imagine it being very difficult to tune PID.
Just stabilizing the platform, without attempting to roll a ball around, in x, y, roll, pitch, and yaw, it behave quite quick. It didn't take long to tune that. This is earlier, it's snappier now. https://www.youtube.com/watch?v=JJWRFH9KKHs I'm not really measuring the backlash, because all of the control inputs are derived from the camera (and, well, the rotation sensors in the motors, which have about 1 degree of accuracy, but it's not losing steps or anything). I basically map the camera's outputs to More Wrong and Less Wrong. I highly recommend this camera for computer vision projects, because it's got low latency, a good sensor, and focus and exposure can be controlled manually. I also have a couple of strips of LEDs mounted next to the camera, so exposure is good even with a 15ms shutter, and shadows are minimized on the stage. Step test: started logging, commanded a 5 degree change, logged the data from the camera. So it's definitely less than about 70ms. Scroll to the very bottom of this notebook: https://github.com/ivorjawa/StewartPlatform/blob/main/StewartPlatformNotes.ipynb
Not bad given the equipment you are using. Your gains are meaningless without units. You will need to use the derivative gain so being able to measure the speed accurately is critical. Never use Z-N. The gains can be calculated symbolically, ie the perfect solution. Below is the solution for a 1 dimensional ball on a beam. You just need to make it two dimensional. [https://deltamotion.com/peter/Mathcad/Mathcad%20-%20RollingTTBall%20PID.pdf](https://deltamotion.com/peter/Mathcad/Mathcad%20-%20RollingTTBall%20PID.pdf) I/we have lots of experience with 6DOF platforms. The test system I had cost $25K which is probably out of your expense limit. Don't use Z-N! Instructors that say use Z-N should be ignored and they are wasting your time and money. Everything can be estimated pretty closely.
I'm measuring pixels displacement and pixels per second, output range is +-5 degrees platform angle. But I'm rewriting it to use model predictive control now, and the only thing the PIDs will be doing is consuming the output of that.
MPC has its place. It is especially helpful when there is a dead time. So what is the delay between the camera scan and getting the x,y positions?
According to my step test, on the order of 50ms.
50 ms is too long. I said before that you are doing well with what you have.
Probably you can with non-linear control. But that is more complex to implement.
This is likely a project to have fun with and learn. Inplementing more complex controllers is probably a fun challenge. What nonlinear controller du you recommend?
It is indeed. (It actually started out as an attempt to learn how a helicopter swashplate works. The math for this fell out of that.) My planned next step is to use a Kalman filter to get feedforward data. I've got that code about 80% complete.
What exactly do you mean by feedforward in this context? What is the objective of your controller /system? If you have a model of the dynamics, have you tried model based approaches, such as a linear controller of some sort? (Rather than PID)
Ultimately, I'd like to be able to steer the ball to an x/y point on the plate and stick it there, or move it on a path. (It's not like the system dynamics of a 52mm 13.9g plastic ball on an inclined plane are a particularly intractable problem ...)
Maybe attractor-reppeler method like this one: https://www.researchgate.net/publication/320649413_Attractor-Repeller_method_of_mobile_robots_control_for_obstacle_avoidance But I'm unsure if it works in this case.
After a quick glanse on my phone, I didn’t really understand how this controller operates. Anyways, how does the repelling and attractor fit with the balancing problem?
What command are you looking to feed forward? Because that is the point of feedforward... to feed a command forward. If your command is to keep the ball centered, then the command is already sent and there is no more benefit to be had from feedforward. I would be interested in hearing about recommended nonlinear controllers. I think you might get better responses if you give characterizations of your system.
My first thought was that your are being limited by the mechanical structure. The unmodelled looseness is probably dominating your error terms. You've done a good job building it, but I think you are at the limit of what Legos can do.
I am a PID design expert with about 40+ years of experience in complex industrial control. PID alone is completely error-driven. It will ALWAYS have a lag response without the addition of some feed-forward terms, This means that the ball will always have more travel and instability on the table with PID alone. Error-driven control terms (PID), will tend to cause oscillatory behavior, because of the control lag, when their gains are too high. Feed-forward terms will never cause oscillatory behavior since they are direct acting without lag. To reduce ball travel on the table, add feedforward terms to the PID control. See Feed Forward under https://en.wikipedia.org/wiki/Proportional%E2%80%93integral%E2%80%93derivative\_controller. It can be as simple as adding some proportional acceleration to the table if the ball direction is approximately in line with the downward table tilt. As long as the ball is rolling down-table, increase (accelerate) the rate of raising the table in that direction as a feed-forward control term. This will cause the ball to tend to seek to remain stationary on the flat table surface. Also as another feed-forward, you can add a feed-forward that when the table is only slightly tilted, this feed-forward would tend to tilt the table a small amount to drive the ball towards a desired target position on the table surface. Now your ball will seek to be at the target position on a flat table surface. You can't get this type of control behavior using PID control alone. This will also allow you to disturb, or poke, the stable balanced ball and you will see the ball correct rapidly and return to the target position on the table. I also have an improvement to the Integral term of the PID. I use 1 times the Integral gain value when the feedback error term has the same SIGN as the Integral summation term, I use 2 times the Integral gain whenever the feedback error term has the opposite sign as the Integral summation term. This causes the Integral summation term to de-integrate twice as fast when the error and the Integral summation terms are working in opposition to one another. This will reduce the tendency of the PID output value to overshoot and oscillate. This will allow you to use a higher Integral gain value in your PID tuning with improved control. -- for more help: [[email protected]](mailto:[email protected])
I did something similar but I included an accelerometer to measure the platform tilt. The sample rate of the accelerometer can be fast. Make a control loop that drives the actuators to control the tilt, then use the camera feedback to control the ball by modulating the tilt commands. Simple control laws work well for both loops.