MICRO-502_Aerial_Robotics_NotesIntro (week1)OverviewFixed wing Staying in the AirMaintaining constant speed VMajor Application FieldsFor AgricultureFor EnergyFor Public safety & securityFor DeliveryCategory«Can Drones Deliver?»Last-cm delivery - Dronistics☑️ CheckpointsMulticopters (week1)IntroductionRotorcrafts (helicopters vs multicopters)Pros and ConsPros-Easy to build and maintainConsStructure and PhysicsMain componentsConfigurationRotation speeds / Forces / MomentsHover conditionsFlight mechanicsMoving Up and DownRotating in YawRotation in Roll/PitchSummary of equationsExample-Translated flightTypes of MulticoptersConfigurationFully Actuated MulticoptersFeaturesProsConsEnergeticsEnergy in hoveringEnergy in forward flight of quadcopterIncrease flight time☑️ CheckpointsAttitude representations (week2)3D Attitude representation- Euler AnglesQuaternionConversionComplex examples✖️ CheckpointsControl (week2&3)Cascaded control ArchitectureControl allocationMethodRemarkRate controlAttitude controlFull quaternion based attitude control for a quadrotorControl strategies for multicoptersLinearExamplesFeaturesPID controllerLQR/LQGNonlinearExamplesFeaturesAn introduction to fully actuated multirotor UAVs☑️ CheckpointsState Estimation (week3&4)Introduction to State EstimationWhy State Estimation in Robotics?Estimation for deterministic systemsState Observer (or Luenberger Observer)Dynamical SystemsEstimation errorState Estimation for stochastic systemsRecap of fundamental conceptsKalman filter (KF)PREDICTIONUPDATEsensor fusion on a quadrotorIntro to EKF, UKF and particle filtersEKFOverviewEquationsUKFParticle filtersState Estimation in aerial roboticsin lab settingsOutdoor☑️ Checkpoints🚧 Navigation (week5)Velocity controlWaypoint NavigationDubins PathsVector fields☑️ Checkpoints🚧 VIO & SLAM (week5)VIOSLAM☑️ CheckpointsFixed-wing drones (week6)IntroductionStructureFlight MechanicsControl surfaceStabilityEnergeticsInduced powerProfile and parasite power☑️ CheckpointsAerial Swarms (week7)IntroReynolds flocking algorithm (Reynolds, 1987)Reynolds flocking: modelReynolds flocking with migrationCase: Aerial swarms for disaster mitigationCommunication radius and turning angleVirtual agents for flocking with fixed-wing dronesReynolds flocking with obstacles (Virtual agents)Other modelsVicsek model: particles in confined environments (密闭环境)Olfati-Saber modelDrone SwarmsVisual information in flockingSoria2019IRC-influence of limited visual sensing using ReynoldsSchilling2019RAL-Learning to flock in simulation with visionSchilling2021RAL-Learning to flock outdoor with vision☑️ Check pointsFlapping-Wing (week8)IntroductionStructureFlight mechanics - Lift generationLift generation in hovering flightAsymmetric hoveringSymmetric hoveringLift generation in forward flightFlight mechanics - ManeuveringEnergetics☑️ CheckpointsDrone Regulations (week8)3 Pillar Concept / Drone CategoriesActSpecific CategoryU-Space☑️ CheckpointsUAS Hardware (week9)IntroductionFrame and materialsmaterials comparisonmetric when considering materialsEnergy sourcesCategoryEnergy and power densityLi-Po batteriesDischarge Curves of Li-Po batteryEnergy Curve of Li-Po batteryActuatorsActuators for propulsionElectric motor example-Brushless DC electric motorsActuators for control/maneuveringServomotorsExamples of ServomotorsPropellersCharacteristicsPitch and efficiency at different cruise speedChoose the right combination actuator and propellerSensorsGyroscopesAccelerometersMagnetometersPressure / Altitude sensorsAirspeed sensorsGlobal positioning system (GPS)Power sensorsOptic flow camerasAutopilotsCommunication protocols✖️ CheckpointsInsect-inspired vision (week10)Optical flowFor pure translational motionSensors used for flight controlArchitecture of insect eyes and brainsElementary Motion Detector 初级运动检测器Experiment - Optomotor Response 视运动反应Wide-field, motion-specific neuronsOptic Flow ComputationGradient Descent MethodsImage Interpolation Algorithm –I2AObstacle avoidance with I2A☑️ CheckpointsAdaptive Morphology in Flying Animals and Drones (week10)Bioinspired Mechanical ResilienceHow do insects cope with collisions?The Size ProblemSelf-deployable origami droneOrigami Drone WingAdaptive Morphology☑️ CheckpointsAgile Flight (week11)Autonomous drone racingDrone acrobaticsLow-latency sensingLearning of flight controllers (week11)challenge1: Architecture/Input and Output representationchallenge2: Data collectionLimitations of Imitation Learning -> RLRL limitationschallenge3: Guarantee the platform's safety during training and testingTakeaways
Lecture notes by Yujie He
Last updated on 2021/07/02
All checkpoints summary can be found here!
Most Civilian Drones are Small
but endurance is a challenge
generate a force Lift L equal and opposite to its own weight W
fixed-wing generates by airflow
computing lift: (sea level)
lift coefficient : proportional to angle of attack
at cruise speed , assuming
Air density: decreases with the altitude
1.25 at sea level
Air speed V: fly 2x as fast -> 4x lift
Wing area S
compute the required velocity
Speed is proportional only to wing loading W/S
generate a thrust force T equal and opposite to Drag force D
Drag
lift-drag ratio
fixed-wing
inspection: large fields; few lights/value
quadcopter
spraying: small fields; high-value crops; difficult terrain
stationary inspection
fire, related to human safety
long-range inspection
frequent, faster inspection to powerline, gasoline
Power generation
strong & steady winds
in-vehicle
policeman
long-range
Border patrol, fast intervention
Forcast: parcel delivery > air freight in near future
Power requirement: 0.59kW
deliver 2 kg payload at cruising speed of 45 km/h
Energy requirement: about 0.39kWh
deliver 2 kg payload within 10 km radius with 30 km/h headwind
Battery & Platform: choose considerations -> cost; weight; energy density; lifetime
Economics: Electricity/Battery cost per km
Packdrone + SimplyFly
Protective foldable cage; Redundant GPS
Temperature-control box
For a given total mass, what type of small drones (multi-copter, fixed-wing, flapping wing) displays the longest endurance?
fixed-wing
fixed-wing drones generate lift with their wings. This means that, unlike a multirotor drone, they don't expend large amounts of energy just to stay in the air and fly more efficiently as a result
How much faster must an intercontinental airplane fly at cruising altitude compared to sea level?
What structural factor (i.e. not the engine) affects the cruising speed of fixed-wing drones?
Speed is only proportional to wing loading (W/S) = Weight/Wing area
What are the two major drone applications in agriculture?
Inspection and spraying
What are the drone applications in the energy sector?
Stationary inspection; long-range inspection; and power generation
What are the factors to consider for calculating the cost/km of drone delivery?
Electricity cost and battery cost
Fixed wing | Flapping wing | rotating wing | |
---|---|---|---|
Examples | airplanes, gliders | new robots | helicopters, multicopters |
Pros | Fast; Efficient | Efficient | Can hover;Highly maneuverable |
Cons | Cannot Hover | Hard to build and control | Less efficient |
Features | Scale down in size | Vertical take-off and landing (VTOL) |
generates lift using high speed rotary blades called rotors
Features
helicopters-Need complex variable pitch rotors
multicopters-Use multiple fixed-pitch blades
overtaken by helicopters due to heavy workload of the pilot
frame; control board; Motors and motor drivers (ESC, electronic speed controller); Propellers; Battery; Receiver
Four propellers generate four lift forces
Propellers 1& 2 (CCW) have opposite pitch compared to propellers 3 & 4 (CW)
movement are controlled by changing the rotation speed of the propellers
All forces must be balanced
move up and down
Lift forces must be parallel to gravity
All moments must be balanced
pitch and roll
Rotor speeds must be balanced (torque balanced)
yaw
How to move a quadrotor around?
Orientation
Violating one or more of these conditions implies that the quadcopter starts to move
Rotor speeds not balanced (torque balanced)
Note: opposite motor pair should increase/decrease motor speeds to keep hovering, or it will keep flying up!
Forces not parallel to gravity
moments not balanced
to control the quadrotor state -> setting the rotor speeds for obtaining a desired angular rotation
inverse operation
Moving Forward
Translated flight requires more thrust than hovering, but not always more power (see section 6)!
pitch down: decrease F_1 and F_2
hover: increase F_1 and F_2 to stop rotation
translate: keep balance between mg and thrust
pitch back
main feature; fully actuated multicopters
Tricopter
More yaw authority compared to quadcopters
direct control of yaw
More complex mechanical design due to the servo in tail
need additional servo to roll/pitch the back motor
Hexacopter/Octacopter
X8 configuration
underactuated multicopters -> all propellers are rotated in the same plane
What is the power consumption of a multicopter during flight?
How to extend multicopters flight time?
Multicopters have high power requirements
inefficient compared to fixed-wing or flapping-wing aircraft
consume about 200 W/kg on average
Centimeter scale quad with LiPo -> 5-7min
Decimeter scale quad with LiPo ->20-30min
changes corresponding to weather conditions and aggressiveness of flight
Power calculation of single motor when drone in hovering
Flight time
Power consumption during forward flight normalized by the power consumption at hover
lower power need but higher thrust -> aerodynamics
tilting the quad to the side -> artificially increase the angle of attack -> generate the increase in thrust in the given power
high power ratio again
drag increases quickly again!
Weight and drag reduction
Increase the specific power of the energy source
switch from LiPo batteries to gasoline (far more energy stored)
Docking station for charging/battery swapping
Tether for power supply
reduce battery
Improving efficiency via mechanical
Energy aware motion planning
reduce acceleration (aggressive flight)
Multi-modal operation
perching; walking and rolling
What set of conditions corresponds to hovering in a quadcopter?
Four conditions: balanced weight; parallel to weight; balanced moment (thrust x half frame); balanced rotation speed
What set of conditions corresponds to a rotation around the yaw axis in a quadcopter?
imbalanced rotation speed
How the time of flight of multicopters can be increased?
How the power consumption and total thrust evolve at different forward speeds in multicopters?
first goes down as thus increases when angle of attack increases
then goes down because the drag force increases again
Rotation matrices can be parametrize by Euler Angles
Yaw-Pitch-Roll rotation matrix (Z,Y,X) or Heading-Pitch-Roll (h-p-r)
can be summarized as (左乘)
🚧 Claw Example
rotation + translation
补充示例!!!
Issues: gimbal lock problem
sensitive to singularities-when pitch angle is 90 degrees, roll and yaw rotations give the same sensor readings
lost 1 DoF
fail to produce reliable estimates when the pitch angle approaches 90 degrees.
can only be solved by switching to a different representation methods, for example quaternions
the union of scalar and vector
rotation vector = quet * vector * quet^(-1)
vector in 3-space
Must meet these requirements
;
multiplication
works for relative to x axis ()
week2_quaternion_mat.png
🚧 Claw Example
rotation + translation using quaternions
frame: image/centered -> camera -> gimbal -> body -> local
🚧 注意谁相对谁!!!
结合exercise进行再次查看
drone architecture
communication: glue to connect different parts (FCU to onboard computer to ground station)
PID controller
Open-loop
Feedforward control
Feedback control
control example
Examples | Free-flight glider | Passively stable helicopter | Racing drone | Autonomous Delivery drone |
---|---|---|---|---|
Sensors | None | None | IMU | IMU + GPS + Vision + Mag |
Controllers | None | None | (Attitude), Rate | Position, Velocity, Attitude, Rate |
Setpoint | None | Manual actuator setpoint | Manual rate setpoint | Autonomous navigation/Manual |
challenges
features
Pros
Decouple translational and rotational dynamics
Facilitate implementation of the controller
implement one by one
Better failure diagnosis capability
modular control architecture for debugging them one by one
Question: why a more advanced controller?
convert thrust & torque setpoint into actuator commands
account for different geometries
from thrust & torque setpoint into actuator
: actuator commands (Nx1)
moment setpoints (3x1)
thrust setpoint
Compute generated force and torque of each actuator
Build matrix "Actuator Effectiveness"
Compute allocation matrix B as the pseudo-inverse of A
not always square, so using pseudo-inverse ()
less effective at producing yaw torque
yaw torque requires more actuator effort than roll and pitch torque
Input body rate setpoint -> Output torque to each angle rate
Input quaternions setpoint -> Output torque to each angle rate
How to compute the quaternion error
axis error
Fresk, E. and Nikolakopoulos, G., 2013, July. Full quaternion based attitude control for a quadrotor. In 2013 European control conference (ECC) (pp. 3864- 3869). IEEE.
without any transformations and calculations in the Euler’s angle space
quaternions
scalar + vector part
Multiplication of two quaternions p, q is being performed by the Kronecker product
represents the combined rotation
quaternion multiplication is non-commutative 不遵循交换律
All quaternions are assumed to be of unitary length 单位长度
The complex conjugate of a quaternion has the same definition as normal complex numbers 复共轭
if the length of the quaternion is unitary then the inverse is the same as its conjugate 逆即是复共轭
Rotation transformation is built by two quaternion multiplications-the normal and its conjugate
rotates the vector v from the fixed frame to the body frame represented by q
where can be rewritten as location in x, y and z axis. For example,
is the rotation axis (unit vector) and is the angle of rotation
Calculating quaternions error dynamics
desired , measured , error --multiplying the reference with the conjugate of the estimated quaternion in Kronecker
Angular errors: the reference is demanding a rotation more than π radians, the closest rotation is the inverted direction and this is found by examining q0. If q0 < 0 then the desired orientation is more than π radians away
其中sgn函数就对应着计算的正负
Pros: Easy to design; Intuitive to tune manually (not necessarily easy on a real drone)
Cons: Imprecise nature of the model due to inaccurate modelling; Limits the performance
Effects of increasing a parameter independently
Features
Practical example-Foldable Drone
Effect of Q and R matrixes
conventional ones have coupling between the horizontal translational and rotational dynamics
fully actuated can help decouple translation and rotation; have direct control to corresponding DoF
key: allocation matrix changes
Fixed-tilt
easier controller; don't need additional motors; lightweight; cannot achieve optimal performance
Variable-tilt: change tilt of propellers to achieve desired motion
can achieve optimal configuration in given task but with complex mechatronics and control
What is the size of the control allocation matrix of an hexacopter with 4 controlled degrees of freedom?
hexacopter 六轴,6motors
Effectiveness matrix: 4x6
Controller allocation: 6x4
Output: 6 motors; input B x (3torque + 1thrust)
最底层的控制,就是输入是torque+thrust,输出是n个电机的转速
Effectiveness: transformation relationship to torque and thrust for each rotor
Why do we use the pseudo inverse instead of standard matrix inverse to compute the control allocation matrix from the actuator effectiveness matrix?
there existing other layout of multicopters except for quadcopters with 4 rotors
What happens when a rate setpoint [0 0 1] is applied?
rotate counterclockwise with 1 rad/s in yaw angle
What are the input and output of the rate controller?
Input: angle rate setpoint + estimated angle rate
Output: generated angle torque
What happens when an attitude setpoint [0 0 pi/2] is applied?
rotate counterclockwise with pi/2 in yaw angle
What are the input and output of the attitude controller in a cascaded architecture?
Input: desired quaternion setpoint + estimated quaternion
Output: generated angle rotation rate
How to compute attitude error quaternion from the estimated attitude quaternion and the attitude setpoint quaternion?
compute quaternion error
calculate angular errors
Perfect model & sensor are not available; Errors exist in both sensors & models
State Estimation on Aerial Robots
Height; Attitude; Angular velocity (Most important as they are the primary variables used for the stabilization of the UAV); linear velocity
main sensors
observer error
=>
with linear model
if A-KC <0, the error will lead to 0 when t -> infinity
How to choose K?
Mean and variance encode the a‐priori knowledge of the random variable
What is the a‐priori information about uncertainty in our problems?
W/V are known but w/v are zero-mean
What is the statistical independence among the values of a random variable in time?
white noise stochastic processes -> 期望值都是0
example: estimate drone’s roll angle
Prediction (A-Priori) + Update
use Gaussians to implement a Bayesian filter. That’s all the Kalman filter is - a Bayesian filter that uses Gaussians
state
error covariance
keep track not only the state but also the covariance (will keep increase)
KF Gain
trust more in model y_k
R-> 0 => K_k = 1/C => x_k = y_k/C
trust more in measurement x_k
P_k -> 0 =>K_k = 0 =>x_k = x_k^{_}
Update state
minimize the Covariance
use multiply KF to achieve better estimation
Predication
F-Computed Jacobian matrix
Update
Near optimal Kalman Filter gain
Combines the results to estimate the mean and covariance of the state
!computationally intensive
Different UKF
What structure does the Luenberger Observer remind you?
closed-loop estimation for deterministic systems
What does the Kalman Filter (KF) gain do?
determine to trust either more on measurement or a-prior/model
For which kind of dynamical systems the KF is useful?
linear continuous system
!cannot work in discontinuous system
What do we need to know a-priori if we want to apply the KF?
state mean and error covariance
What are the advantages/disadvantages of the UKF w.r.t an EKF?
EKF
What are the advantages/disadvantages of a particle filter w.r.t an UKF?
PF
Which component of the acceleration setpoint is converted to roll angle? To pitch angle?
Why is navigation based on fixed position setpoint inappropriate for fixed-wing drones?
What type of segment compose a Dubins path?
In which case can we construct a RLR Dubins path?
distance of waypoints is less than the minimal diameter
What happens when centrifugal acceleration is not taken into account in the formulation of a circle following vector field?
It will not follow the vectors and cross the circle lines
What is VIO/SLAM useful for?
VIO: Visual inertial odometry
process of estimating the drone state (position, orientation, and velocity) by using only the inputs from Camera(s) and IMU(s)
SLAM: Simultaneous Localization and Mapping
What are the sensors that can be used to do VIO/SLAM?
Camera(s) and IMU(s)
What is loop closure?
the recognition of when the robot has returned to a previously mapped region and the use of this information to reduce the uncertainty in the map estimate.
What are the main three VIO paradigms?
Filters
Fixed-lag smoothers
Full smoothers
Why do we use a camera and an IMU for VIO?
Cheap and complementary to each other
Camera: Precise during slow motion and provide rich information
Limited output rate; Not robust to scenes with low texture, high speed motions
IMU: Scene independent and High output rate
Poor signal-to-noise ratio at low accelerations and angular velocities; Estimated motion tends to accumulate drift quickly
Ailerons
elevators
rudder
longitudinal -> pitching
lateral -> rolling
directional -> yawing
What are the main flight control surfaces in fixed-wing aircrafts?
Ailerons
elevators
rudder
What are the main stabilization strategies in fixed-wing aircrafts?
Longitudinal stability
making CoG ahead of the aircraft
Lateral stable
Vertical stability-pitch
making the side surface greater than ahead of the center of gravity
How is a flying wing/Tailless aircraft controlled and stabilized?
What are the main contributions to power consumption in fixed-wing aircrafts?
Pressure drag + Friction drag
Induced drag + Profile drag (wing) + Parasite drag (body)
Induced drag(powers proportional to 1/v)
drag on the wing that is due to lift
Profile drag, Parasite drag (powers proportional to v^3)
Drone light shows
Centralized = agents transmit individual position to ground computer and receive next location
Collective Motion in nature
Decentralized = agents rely on local information and computation
Equations
Equilibrium
Positions converge to a lattice formation (晶格式)
Velocities converge to the average of initial velocities
new migration rule steers the swarm towards a desired direction
Equation
,
parameters
SMAV platform with control electronics
Winged drone flies around Virtual Agent which moves according to Reynolds rules
Varga et al., Distributed Formation Control of Fixed Wing Micro Aerial Vehicles for Uniform Area Coverage, IROS 2015
video: https://youtu.be/FYsd2VckGA0
Obstacles are modelled as virtual agents
position and velocity of the virtual agent
Virtual agents exert separation and alignment effects, but not cohesion (not collide with the agent)
Visualization
Equation (two extra separation and alignment term regarding obstacles)
Vasarhelyi et al., Optimized flocking of autonomous drones in confined environments, Science Robotics, 2019
DOI: http://doi.org/10.1126/scirobotics.aat3536
Video: https://youtu.be/E4XpyG4eMKE
Project web: http://hal.elte.hu/drones/scirob2018.html
Rules
Equation
The full equation contains 12 parameters and requires heuristic methods for optimization
R. Olfati-Saber, Flocking for multi-agent dynamic systems: algorithms and theory, IEEE Transactions on Automatic Control, 2006
Rules
Distance matching
Alignment: attempt to match the velocity and direction
Equation
distance matching example
Components
Note
Coppola et al., A Survey on Swarming With Micro Air Vehicles: Fundamental Challenges and Constraints, Front. Robot. AI, ‘20
The combination of centralized planning/control with external positioning has allowed to fly significantly larger swarms. The numbers are lower for the works featuring decentralized control with external positioning, or centralized control with local sensing
Three categories
Centralized with external positioning
latest: September 20 2020
3,051 drones
News: https://www.guinnessworldrecords.com/news/2020/10/3051-drones-create-spectacular-record-breaking-light-show-in-china (Company: https://www.dmduav.com/)
YouTube: https://youtu.be/44KvHwRHb3A
Decentralized with external positioning or centralized with on-board sensing
Vasarhelyi et al. (2019)
Decentralized with on-board sensing
Saska et al. (2017)
Soria et al., The influence of limited visual sensing on the Reynolds flocking algorithm, 2019
generate flocks with different fields of view
measure flocking performance (all individuals in the flock have the same visual configuration)
results
Schilling et al., Learning Vision-Based Flight in Drone Swarms by Imitation, RAL2019
Stages
Note
Schilling et al., Vision-Based Drone Flocking in Outdoor Environments, RAL2021
Setup
Control method
Real-time drone detection
Input: images from 4 cameras
Output: x,y coordinates of perceived drones in image frame coordinates
known size to compute corresponding distance
Multi-agent state tracking
Potential-field-based control
What information does each agent receive in the Reynolds flocking algorithm?
position and velocity of self and neighbor agents
How are obstacles modeled in Reynold’s flocking
virtual agent; integrate into equations with alignment and separation term (non cohesion)
How is a migration point incorporated in flocking algorithms
add a migration velocity term
What does the Olfati-Saber algorithm ensure?
No collision. The acting force will be zero when reach the preferred distance
What are the three steps of vision-based drone flocking algorithm?
images from 4 cameras -> x,y coordinates of perceived drones in images -> Range and bearing of all perceived drones -> velocity vector
lift and thrust generation, and maneuvers mostly obtained by using the wings
imitate the flapping-wing flight of birds, bats, and insects
scale down better then rotary crafts and fixed wing UAVs
Challenges:
categories were determined to be the tail (1) and wing design (2)
Flapping Wings
two wings are flapped to produce both lift and thrust, thus overcoming gravity and drag to provide sustained flight
Clapping
Increase of lift during the “clapping
cancels out the vertical oscillations
Morphing/folding
flap their wings downward, fold them in toward their body
minimum wing area during the upward flap -> minimize undesired negative lift
produced during the entire wing stroke exploiting different unsteady mechanisms.
Downstroke/Upstroke
flapping wing MAVs can use the tail and / or the wings for control.
ail actuation
Wing actuation
What are the main types of tail designs found in flapping wing MAVs?
What are the main types of wings designs found in flapping wing MAVs?
What are the main mechanisms for lift generation in symmetric hovering flight?
What are the main steering strategies in flapping flight MAVs?
Tail actuation
Wing actuation
Author: Markus Farner
https://www.bazl.admin.ch/bazl/en/home/good-to-know/drohnen.html
Rules in Aviation: Federal Office of Civil Aviation Switzerland
Everything which is not forbidden is allowed -> Switzerland
Trust, less difficult for innovation
Ordinance on Special Category Aircraft
DETEC Ordinance on Special Category Aircraft
ANSP (Skyguide) or Airport responsibility
Act in EU
Open/Specific/Certified
Difference
Application for an operating permit on the basis of the SORA (Specific Operations Risk Assessment)
Operational Volume = Flight Geography + Contingency Volume
The U-space is a collection of decentralized services that collectively aim to safely and efficiently integrate drones into the airspace and enable drone operations alongside manned flight.
https://www.bazl.admin.ch/bazl/en/home/good-to-know/drohnen/wichtigsten-regeln/uspace.html.html
https://www.skyguide.ch/en/events-media-board/u-space-live-demonstration/
airspace in block to avoid collision and report the location for further path calculation
Federal Office of Civil Aviation FOCA
What defines the three drone categories?
Open-within the legal framework (No Authorization required)
low risk; maximum flying altitude: 120m
Specific-Not sufficiently safe (Authorization required)
enhanced risk
Certified-Approved to accepted standards
risk comparable to manned aviation
What is the SORA declaration?
the multi-stage process of risk assessment aiming at risk analysis of certain unmanned aircraft operations, as well as defining necessary mitigations and robustness levels
Is FOCA authorization sufficient for operating drones in the “Specific” category?
No. FOCA authorization is required for the "Specific" category, but it is not sufficient because other federal, cantonal, and communal authorities may require additional authorizations
What is U-space?
🚧 U-Space provides a framework to facilitate the implementation of all types of operation in all classes of airspace and all types of environment, while ensuring an orderly coexistence with manned aviation and air traffic control.
from U-Space: The airspace of the future
main component required
The aerial vehicle
Air frame
Actuators for propulsion and control
Energy source
Autopilot
Payload
Ground Control Station
Material | Composite | ABS/PLA | Wood | Foam |
---|---|---|---|---|
Pros | Stiff, lightweight | Easy to manufacture by 3D printing or injection molding | Lightweight and cheap | Lightweight and soft, resistance to collision |
Cons | Expensive, complex to manufacture | Heavier, less stiff | complex to work with | limited load |
Comment | - | useful for prototyping | - | absorb energy, less prone to damage |
Young's modulus [wiki]
弹性模量,正向应力与正向应变的比值
Specific modulus [wiki]
比模量,单位密度的弹性模量,劲度-质量比,在航天工业中有广泛应用。
Goal: power the robots to fly
Metric: energy density, power density, charging time and so on
Nickel-Cadmium (NiCd) | 镍镉
Nickel-Metal Hydrate (NiMh) | 镍氢电池
由镍镉电池(NiCd battery)改良而来的,其以能吸收氢的金属代替镉(Cd)。它以相同的价格提供比镍镉电池更高的电容量、较不明显的记忆效应、以及较低的环境污染(不含有毒的镉)
[wiki-zh]
Lithium-Polymer (Li-Po) | 锂离子聚合物电池
Fuel
Fuel cell
energy density
amount of energy stored per unit volume or mass
power density
how fast or quickly to discharge into mechanics
amount of power (time rate of energy) per unit volume or mass
Conclusion
most commonly-used UAV energy source
Each battery composed of one or more cells connected in series
S=series, P=Parallel
Each cell has
nominal voltage of 3.7 V
a maximum voltage of 4.2 V
a capacity (mAh)
e.g., 1000 mAh
a specific discharge and charge rate (C)
e.g., Discharge rate with 25-50C = 25-50 A of max continuous discharge current; Charge rate 2C = 2 A
Book: G. C. H. E. Decroon, M. Perçin, B. D. W. Remes, R. Ruijsink, and C. De Wagter, The delfly: Design, aerodynamics, and artificial intelligence of a flapping wing robot. 2015.
How much energy the same LiPo battery can provide until its voltage drops below a certain voltage
10 times higher battery load (discharge rate) -> 17 times shorter flight time
nonlinear relationship
Electric motors | Combustion engine | Hybrid | |
---|---|---|---|
Pros | clean and quite; Reliable and easy to maintain; Fast to change operational state (accelerate and decelerate) | High weight to power ratio using fuel | Long endurance; Suited for fast change of speed |
Cons | Limited weight to power ratio due to battery | Vibration, dirt, and noise; Requires tuning; Not suited for fast change of speed | Complex and expensive |
Combustion engine is not suited for fast change of speed (problem in controlling quadcopters)
Hybrid systems (fuel generator coupled with electric motor)
e.g. skyfront drone with 4.5 hour endurance (demonstrated) and 3 kg payload capacity
Brushless: no electrical physical connection
Pros
Cons
Main motor data
3 primary data:
Size
Nominal voltage (number of battery cells, e.g., 3S)
Speed constant KV (No load rpm/Volt)
need to deflect the control surfaces
3 wires (B-Ground, R-Voltage, Y-Signal) - send power and signal to control circuit
brush motor in small scale and connected to gear drive (set correct speed and torque, connect to potentiometer)
potentiometer (电位器) sensor for angular position control
Rotary servos with push rod | Linear servos |
---|---|
![]() | ![]() |
Weight: 1 to 500 g | Weight: 1 to 5 g |
- | to control elevators, flaps and ailerons |
to convert power (delivered by a rotating shaft) into thrust
Diameter
Pitch
Number of blades
the blade pitch could be varied in flight
propeller advance ratio VS Propeller efficiency
, flight speed , angular speed , and Diameter D -> tip speed
choose the suitable propeller according to the diameter and pitch to achieve better efficiency curve
Variable pitch propeller with servo -> achieve best efficiency all the time
match the propeller and the motor to maximize propulsive efficiency
Proprioceptive sensors: measure the internal state of the UAV, mainly for control
Exteroceptive sensors: provide information about the UAS environment and are usually carried as a payload
measure changes in vehicle orientation
Type: Mechanical; Optical; Micro-electromechanical systems (MEMS)
Categories
Cons
all gyroscopes are prone to drift unless the error is corrected through reference to some alternate measurement
(not relative to absolute reference but past state)
the drift will eventually exceed the required accuracy
MEMS rate gyros
vibrating mechanical elements to sense Coriolis acceleration (振动机械元件以感应科里奥利加速度)
induce an vibration outside the plane and measure the out-of-plane motion
Pros -> replacing mechanical and optical gyros
measure acceleration to get the inertial information
Exteroceptive
electronically compass 电子罗盘
direct measure of the magnetic field
Hall-effect (霍尔效应)
Flux Gate (磁通罗盘) [wiki]
two perpendicular circuits to get the force
Pros
weakness of the Earth magnetic field
easily disturbed by magnetic objects or other sources
not working in indoor environments
because of wires or other electronic device
to measure the altitude according the atmosphere pressure
measure the changing distance of the deforming membranes: piezoresistive (压阻式), capacitive, optical, electromagnetic, etc
has a vacuum inside the housing to get an absolute pressure
measured using a pitot tube (皮托管)
directed into the direction of motion
the difference between the stagnation pressure (static + dynamic pressure) -> the airspeed
measures the speed of a UAV with respect to the air (airspeed) -> used for fixed-wing UAV
not the absolute speed of the UAV
Global Navigation Satellite System (GNSS): This term includes
Relatively lower accuracy: have a position accuracy within 20 m in the horizontal plane and 45 m in the vertical plane
enhancement techniques
WAAS or other ground tower-based services: static
get close to 1-2 m accuracy
Real time Kinematic (RTK) positioning: Base Station receiver and a receiver on the vehicle
close to 1 cm accuracy
system used to stabilize (e.g. attitude stabilization of a multicopter) or to control the trajectory
receive the input information -> process information -> send actuator commands
, where
motion parallax, OF is
Experiments1
Experiments2
Compound eyes: a large set of eyes to detect in different directions, no color capability -> used to detect optical flow
Ocelli: a small set of eyes are sensitive to luminosity to detect contrast; on the head and point upward -> used for stabilization, orientation, and attitude
Halteres (like accelerators) -> used to measure rotational speed and stabilize than visual information
Optic flow is detected by neurons in the lamina (椎板), whose response is aggregated and transformed by neurons in the medulla (髓质) and in the lobula plata (小叶平台) of brain regions
Correlation between two adjacent, time-delayed contrast detectors | 两个相邻的延时对比检测器(小眼)之间的相关性
photo receptor -> temporal delay -> correlation -> subtraction
the speed of motion can be detected as the peak the motion
not a reliable velocity estimator -> depends on temporal and spatial frequency -> cannot measure velocity objectively
torque response is not related to optical flow speed
assumption: brightness I does not change across the image (n,m) as the agent moves over time t
Handcrafted Example: Lucas-Kanade method
often iterative and requires significant computing power
computed as the image shift that generates the smallest error between artificially shifted versions of the image at time t and the image at time t + Δt
used in Crazyflie drone for optical flow calculation
two cameras to look to left and right with DoF of 40 degrees
use cameras (optical flow) and gyro with mechanisms (rudder方向舵/elevator升降舵) to control more DoF systems
Zufferey, Klaptocz, Beyeler, Nicoud and Floreano (2007), Advanced Robotics
control pitch and roll of fixed-wing drones by building relation between optical and control planes directly
A. Beyeler, J.-C. Zufferey and D. Floreano (2009) Autonomous Robots, 27(3), 201-219
Making an Artificial Compound Eye
Floreano, Pericet-Camara, Viollet et al, PNAS, 2013
Influence of agent’s rotation and translation on optic flow and distance estimation
Influence of angular velocity, spatial frequency, and temporal frequency on EMD (elementry motion decoder)
angular velocity and temporal frequency都是先增后减。
spatial frequency 越大,角速度的图越向后偏移
Functioning of Image Interpolation Algorithm
generate optical flow by computing image shift to minimize the overlap error
use derivation to minimize the overlap error
Methods for discounting rotational optic flow
用IMU估计角速度,然后算rotational optic flow再减去图片得到的即可
remove the rotational optical flow by using IMU to measure yaw angle
drones hit to obstacle and fall into the ground will lead to damage
inspired from insects
Sturdy yet flexible exoskeleton | 坚固而灵活的外骨骼
Dual stiffness wing | 双刚度翼
can bend with surface
Mintchev, de Rivaz, Floreano, Insect-Inspired Mechanical Resilience for Multicopters, IEEE Robotics & Automation Letters, 2017
Frames could
An active uprighting mechanism for flying robots, Klaptocz et al., IEEE Transactions on Robotics, 2012
Briod, Kornatowski, Zufferey, Floreano, A collision‐resilient flying robot, Journal of Field Robotics, 2014
flyability
Mintchev, Daler, L’Eplattanier, Saint_Raymond, Floreano, Foldable and self-deployable pocket sized quadrotor, ICRA, 2015
Dufour, Owen, Mintchev, Floreano, A drone with insect-inspired folding wings, IROS 2016
Daler, Lecoeur, Hählen, Floreano, A flying robot with adaptive morphology for multi-modal locomotion, IROS Proceedings, 2013
A bioinspired multi-modal flying and walking robot, Bioinspiration & Biomimetics, 2015
can use wing to fly and rotate wing to walk on the ground
Air and Ground Locomotion
Ajanic, Feroshkan, Mintchev, Noca, & Floreano (2020) Science Robotics
Morphing Wings to adapt different winds
LIS hawk inspired from northern goshawk
Maneuverability
able to change the velocity vector by changing the wing status
higher linear accelerations on body frame/higher lift/drag coefficient
Agility
able to change angular rate, e.g. pitch and roll angles
Higher angular rates are produced by larger pitch and Agility roll coefficients by extending the wing or not
Power requirement and speed range
Extending wing and tail increases lift and allows lower speed range
Strategies for collision resilience in flying animals and robots
inserts
Robots
Trade-offs between aerial and ground locomotion that require adaptation
Actuator torque and speed
Fly require high speed and ground locomotion requires high torque. Few motor can satisfy both.
Mass of center
Fly: a bit front , Ground walking: in the center
Inner ratio
约等于wing span,飞的时候展开,地上收起来
Effects of wing and tail span on lift and drag coefficients, and on required power
Extending for aggressive flight while Tucking wing (折翼) for cruise flight
Tucking need base speed to activate
Pfeiffer, Scaramuzza (2021) Human-piloted drone racing: Perception and control, RAL’21. PDF. Dataset.
Humans focus visual attention on future waypoints: receding planning horizon
220 ms perception-control latency → humans can definitely be beat by a machine in a speed race
try to plot literature in different axis (External sensors & agile)
Research challenges & Opportunities
P. Foehn et al., AlphaPilot: Autonomous Drone Racing, RSS 2020, Best System Paper Award. PDF YouTube
pass a sequence of gates as soon as possible
[1] Foehn, Scaramuzza, CPC: Complementary Progress Constraints for Time-Optimal Quadrotor Trajectories, arXiv preprint, 2020. PDF. Video.
Falanga, Foehn, Peng, Scaramuzza, PAMPC: Perception-Aware Model Predictive Cotrol, IROS18. PDF. Video. Open Source: https://github.com/uzh-rpg/rpg_quadrotor_mpc
action objectives + perception objectives => optimization problem
Kaufmann et al., Beauty and the Beast: Optimal Methods Meet Learning for Drone Racing, ICRA’19. PDF. Video Deployed to win the IROS Autonomous Drone Racing Competition, 2018. Video.
NN to detect the gate (robust to motion blur and illumination variation)
Loquercio, et al., Deep Drone Racing: From Simulation to Reality with Domain Randomization
predict gate pose and covariance
trained in simulation only
sim-to-real world transfer via domain randomization!!!
can be used to drone approach
gazebo simulator was used
ACADO (http://acado.github.io/). ACADO's C++ interface is used to describe the quadrotor model and parameters for transcription into a quadratic program, which is then solved with qpOASES (https://projects.coin-or.org/qpOASES).
Kaufmann, Loquercio, Ranftl, Mueller, Koltun, Scaramuzza, Deep Drone Acrobatics, RSS 2020. PDF. Video.Code Best Paper Award Honorable Mention
traditional drone control architecture
Image/IMU -> State estimation (VIO) -> Planning -> Control -> Collective thrust/Bodyrates -> Low-level controllers
End-to-end sensorymotor control (in a concurrent design fashion)
Image/IMU -> NN -> Collective thrust/Bodyrates
Zero-shot Sim2Real Transfer
related literature
Does computer vision matter for acion? Science Robotics
Controller experiments indicate that explicit intermediate representations help action
use Feature tracks into the NN tracking
Simulation results: lower tracking errors and 0 errors in acrobatics
Cons: does not generalize to different tasks
event cameras do not suffer from latency/motion blur or illumination variation.
Application
imitate and use data from cars and bicycles
DroNet: Learning to Fly by Driving, Loquercio et al., Robotics and Automation Letters 2018, PDF, Video, Code.
Simulation to Reality Transfer -> Sim2Real: Domain Randomization for Drone Racing
Deep Drone Racing with Domain Randomization
Loquercio et al., Agile Autonomy: Learning High-Speed Flight in the Wild, Under Review
Transfer learning: Abstraction
How to find the abstraction function
Imitation learning by using current SOTA method (RRT , A*, MPC, LQR) as GT
Design a reward function, then learn via interaction
difficult to design the reward function
state space
vision-based state space is huge -> hard to guarantee enough coverage of the state space
Loquercio et al., A General Framework for Uncertainty Estimation in Deep Learning, RA-L 2020