104
edits
2023-2024 Qubo Software Projects: Difference between revisions
→Vision
(controls) |
(→Vision) |
||
|
== Controls ==
<blockquote>Note: This section was copied directly from the old wiki. This page is intended to be approachable for new members transitioning to working on Qubo. I'm not sure how understandable this is, as it's not my area; so just please don't avoid a project you're interested in because the description sounds scary. For questions and concerns about these projects, ask in #team-software in the Slack. --- Jeffrey Fisher</blockquote>Controls as in [[wikipedia:Control_theory|"control theory"]]. Making Qubo be able to stand still (to fire torpedos accurately), move efficiently, etc.
=== Goals ===
== Vision ==
<blockquote>Note: This section was copied directly from the old wiki. This page is intended to be approachable for new members transitioning to working on Qubo. I'm not sure how understandable this is, as it's not my area; so just please don't avoid a project you're interested in because the description sounds scary. For questions and concerns about these projects, ask in #team-software in the Slack. --- Jeffrey Fisher</blockquote>
=== Goals ===
# Detect competition objects accurately and efficiently (minimize computational cost)
# Determine objects' relative poses (relative linear position/coordinates and rotational orientation)
=== Current state ===
* YOLOv8 model trained on [https://app.roboflow.com/josh-smith/robosub2023training/overview our data] and [https://app.roboflow.com/22ghoshi/cw-akqjp/deploy/2 some other team's data] in [https://colab.research.google.com/drive/1jiaf1PEvYl7cbJP1DTVHJZ2XlTB61Jgj?usp=sharing this notebook] very quickly during competition
=== Things to work on ===
* What ML model(s) do we want to use? (my knowledge is quite limited)
** YOLOv8 worked pretty well for identifying objects considering how rushed our training and implementation was
** Are there models that are feasible for us that can give us more info than YOLO, maybe something like [https://arxiv.org/abs/1711.00199 PoseCNN] that gives full pose estimation given 3D models of the objects?
* If we use YOLOv8, one of the big issues was speed, both initialization (took like 1 minute at competition) and detection speed
** Likely can be easily improved just by using the proper format for the Jetson, right now we're just using .pt but [https://wiki.seeedstudio.com/YOLOv8-TRT-Jetson/ TensorRT (tutorial)]/[https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#overview TensorRT (docs)] is recommended, we can test other formats using their [https://docs.ultralytics.com/modes/benchmark/ benchmark]
** Also can use quantization args for [https://docs.ultralytics.com/modes/predict/ predict] and [https://docs.ultralytics.com/modes/export/#arguments export]
* Data collection - we collected all of our data at competition (including yoinking another team's dataset)
** Some other teams constructed 3D models in Unity/other graphics engines and trained on those, is that worth it?
** Ideally we can have a baseline model trained on data we either synthetically create or we collect in the pool using physical recreations of the objects, before we go to competition, that we can train off of
* Traditional CV - if our ML stuff only gives us the bounding box of the object in the image, can we use traditional CV stuff to get the relative pose of objects?
** E.g. algorithms like SIFT (feature matching), contour/corner detection -> solvePnP, etc.
* Make labeling easier - labeling was an incredibly massive time sink during competition
** Roboflow hides model-assisted labeling behind a paywall, we found [https://anylabeling.nrl.ai/docs/custom-models this cool open source thing] that lets you do model-assisted labeling pretty easily
== GUI ==
| |||