Skip to content

Add acquisition automation modes and acqmode command#391

Draft
cfremling wants to merge 75 commits intomainfrom
CF/auto-acq
Draft

Add acquisition automation modes and acqmode command#391
cfremling wants to merge 75 commits intomainfrom
CF/auto-acq

Conversation

@cfremling
Copy link
Collaborator

@cfremling cfremling commented Feb 1, 2026

This adds 3 mode settings for acquisition with various levels of automation.
mode 1 is identical to current (legacy acquisition)
mode 2 waits for seq continue between each step (goes through acquisition, fine tuning using scam, offset, expose)
mode 3 is full auto

the fine tuning step uses ngps_acq (now embedded in slicecamd)

needs comprehensive on-sky testing before pull approved

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a89f02d2d3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines 593 to 595
auto wait_for_guiding = [&]() -> long {
ScopedState wait_state( wait_state_manager, Sequencer::SEQ_WAIT_ACQUIRE );
this->async.enqueue_and_log( function, "NOTICE: waiting for ACAM guiding" );

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid clearing SEQ_WAIT_ACQUIRE in wait_for_guiding

In auto modes 2/3, wait_for_guiding creates a ScopedState for SEQ_WAIT_ACQUIRE, which clears that bit as soon as the helper returns (on guiding, timeout, or error). The acquisition thread already sets the same SEQ_WAIT_ACQUIRE bit in dothread_acquisition, so this new scoped state can clear the bit while acquisition is still running. Any clients or logic watching the wait-state (telemetry/UI or other waits) will see acquisition complete early and may proceed to expose even though acquisition is still active. Consider removing this ScopedState or using a separate state/refcounted mechanism for the “waiting for guiding” phase.

Useful? React with 👍 / 👎.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will add a new state to wait for inside the mode loops to not mess with the overall acquisition state

Copy link
Collaborator Author

@cfremling cfremling Feb 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Added SEQ_WAIT_GUIDE instead of SEQ_WAIT_ACQUIRE while waiting
    for guiding.
  • Added ACQ_FINE_TUNE_XTERM so fine-tune can run in its own xterm when
    enabled so it can be properly monitored.
  • Fine-tune process is now launched in its own process group and is terminated on seq abort . works even if seq abort sent when ngps_acq in progress

seq acqmode 1,2,3 TESTED with NGPS simulator mode

Commit: a20b509

astronomerdave and others added 17 commits February 2, 2026 08:21
sequencerd is subscribing to that and waiting on that to start an exposure
updates sequencerd.cfg.in
other minor cleanup
- Add acqmode command to control acquisition automation (modes 1-3)
- Mode 1: legacy manual mode (wait for user at each step)
- Mode 2: semi-automatic (start acquisition, wait for user to expose)
- Mode 3: fully automatic (apply offsets automatically)
- Add repeat target detection to skip unnecessary slew/acquisition
- Add configuration parameters for acquisition automation
- Skip acquisition when re-observing same coordinates (modes 2-3)

Based on PR #391 acquisition automation features
Applied to DH/sequencer-calib-work branch (observatory baseline)
- Qt Widgets GUI for managing target sets and targets
- Direct database access via MySQL Connector/C++ (X DevAPI)
- Features: table views, inline editing, drag/drop reordering
- Search functionality and sequencer controls
- Build with CMake, reads DB settings from Config/sequencerd.cfg
- Add seq-progress launch script and X11 GUI (utils/seq_progress_gui.cpp)
- Implement seq_progress topic publishing in sequencer
- Track fine-tune, offset, and target state changes in real-time
- Publish progress at key workflow points:
  * Before/after fine-tuning (tracks fine_tune_pid)
  * Before/during/after offset operations (tracks offset_active flag)
  * On target activation and completion
- Add seq_progress topic and keys to common/message_keys.h
- Add offset_active atomic flag to track offset state
- Update utils/CMakeLists.txt to build seq_progress_gui with X11

This enables full 5-phase progress visualization (SLEW → SOLVE → FINE → OFFSET → EXPOSE)
for on-sky monitoring and testing of acquisition automation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The seq_progress_gui.cpp includes common.h which needs logentry.h from utils directory.
Add PROJECT_UTILS_DIR to target_include_directories to resolve build error.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move utilities library after logentry and network in link order, since
those libraries depend on symbols from utilities. Linker requires
dependencies to be specified after the libraries that use them.

Fixes undefined references to: tmzone_cfg, timestamp_from, get_time,
get_latest_datedir, is_owner, has_write_permission, strip_control_characters

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CalibrationTarget::get_info() returns a const reference (not a pointer)
and throws exception if not found. Updated calib_set() to:
- Remove unnecessary null check (get_info throws exception instead)
- Use '.' instead of '->' to access calinfo members (it's a reference)
- Store as 'const auto &' to match return type

Fixes compilation errors in sequence.cpp:2332, 2341, 2342, 2353, 2380

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update MySQL Connector/C++ paths in ngps_db_gui CMakeLists.txt to search:
- /include (not bare )
- /lib64 (where library actually is on production)
- Prioritize mysqlcppconn8 to match main build

This matches the paths used successfully by utils/CMakeLists.txt

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
cfremling and others added 14 commits February 8, 2026 00:39
- Remove exposure_progress > 0.0 check so percentage always shows
- Show 'EXPOSURE 0%' immediately when phase becomes active
- Add frame count for NEXP>1: 'EXPOSURE 2/3 12%'
- Ensures percentage is visible even when starting mid-exposure
- Print received EXPTIME values to stderr
- Show exposure_progress calculation
- Help diagnose why percentage stays at 0%
- Temporary debug output to be removed once issue resolved
- Remove percentage display (UDP EXPTIME not reliably received)
- Show 'EXPOSURE X/Y' for NEXP>1
- Show 'EXPOSURE' for NEXP=1
- Keep EXPTIME parsing code for future use
- Remove debug logging
- Show percentage on EXPOSURE label again
- Debug: Print EXPTIME messages received via UDP
- Debug: Print parsed values (remaining, total, percent)
- Debug: Print exposure_progress after update
- Debug: Print label being drawn every 10 frames
- Will help diagnose if UDP is received but not parsed/displayed
- Print whether UDP listener will be started
- Print msgport and msggroup from config
- Print UDP listener file descriptor if started
- Help diagnose why EXPTIME messages not received
Sequencer:
- Parse EXPTIME UDP messages and republish to seq_progress ZMQ topic
- Rate-limit: only publish when percent changes (reduces message spam)
- Format: exptime_remaining_ms, exptime_total_ms, exptime_percent

GUI:
- Receive exptime_percent from seq_progress ZMQ topic
- Apply smoothing (70% old, 30% new) to percentage
- Works when UDP listener disabled (use_udp=0, msggroup=NONE)
- Fixes issue where GUI shows 0% during entire exposure
- Set NGPS_ROOT=/home/developer/Software directly
- Remove environment variable fallback logic
- Simplifies launch: just run 'seq-progress' without setting NGPS_ROOT
- Find script's own directory using BASH_SOURCE
- Set NGPS_ROOT to parent directory of script location
- Works from any current working directory
- Works for both /home/developer/Software and NGPS_tmp installs
- Use 'export' so child process inherits NGPS_ROOT
- Fixes 'ERROR: opening configuration file Config/acamd.cfg'
- seq_progress_gui needs NGPS_ROOT to find acamd.cfg for ACAM port
When waiting for user continue after acquisition/fine-tune failure,
the button now shows "OFFSET & EXPOSE" or "EXPOSE" depending on
whether the target has a database offset. A blinking red instruction
message tells the user exactly what clicking the button will do.

The sequencer already applies target offset before exposure in both
the guiding-failure and fine-tune-failure paths; the wait messages
are updated to reflect this ("apply offset and expose").

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sends 'native z' via tcsd before target_offset() in both the
guiding-failure and fine-tune/post-fine-tune paths. This resets
cumulative offsets from acquisition and fine-tuning so the observer
sees only the intended target offset.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…d processes

When sequencerd forks child processes (e.g. ngps_acq), the listening socket
file descriptors were inherited by the child. If the parent was killed while
a child was still running, the child held the port open, preventing restart.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Publish acq_automatic_mode in sequencer ZeroMQ progress messages and
show it as a right-aligned label in the GUI title bar (MANUAL/SEMI-AUTO/AUTO).
Also poll acqmode via TCP fallback for robustness.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@astronomerdave
Copy link
Contributor

Cannot approve

This PR introduces process management (fork/exec, waitpid, signal handling) directly into the sequencer daemon, which is inconsistent with the project's architecture.

The NGPS system is built around:

  • Separate daemon processes communicating via message passing
  • Asynchronous command/response patterns
  • Thread-based concurrency without direct process management

This PR breaks those patterns by:

  • Fork/exec'ing external programs (ngps_acq) and managing their PIDs
  • Adding synchronous blocking syscalls (waitpid) inside async contexts
  • Introducing signal handling (SIGTERM/SIGKILL)

The system lockup you experienced is likely due to these blocking syscalls interfering with the multi-threaded daemon architecture - deadlocks are almost inevitable when mixing low-level process management with the existing async design.

This PR is also relying upon an external C program whose code is not part of the repository.

Recommendation:
To implement this functionality into NGPS it should likely be implemented into the acam daemon, perhaps something along the lines of the draft PR #363 that was started for this work.

The approach of shelling out to external commands doesn't fit the NGPS architecture and creates significant reliability risks.

@astronomerdave astronomerdave marked this pull request as draft February 10, 2026 17:06
@cfremling
Copy link
Collaborator Author

cfremling commented Feb 10, 2026

Cannot approve

This PR introduces process management (fork/exec, waitpid, signal handling) directly into the sequencer daemon, which is inconsistent with the project's architecture.

The NGPS system is built around:

  • Separate daemon processes communicating via message passing
  • Asynchronous command/response patterns
  • Thread-based concurrency without direct process management

This PR breaks those patterns by:

  • Fork/exec'ing external programs (ngps_acq) and managing their PIDs
  • Adding synchronous blocking syscalls (waitpid) inside async contexts
  • Introducing signal handling (SIGTERM/SIGKILL)

The system lockup you experienced is likely due to these blocking syscalls interfering with the multi-threaded daemon architecture - deadlocks are almost inevitable when mixing low-level process management with the existing async design.

This PR is also relying upon an external C program whose code is not part of the repository.

Recommendation: To implement this functionality into NGPS it should likely be implemented into the acam daemon, perhaps something along the lines of the draft PR #363 that was started for this work.

The approach of shelling out to external commands doesn't fit the NGPS architecture and creates significant reliability risks.

I do agree that spawning a new process for acq isnt ideal. The ngps_acq communicates with acamd, sequencerd, and slicecamd. As such, I think a new daemon, acqd could make sense, but probably overkill. I think slicecamd is the place for this not acamd since its using slicecam image data

@cfremling
Copy link
Collaborator Author

@astronomerdave I added ngps_acq.c to slicecamd/ and got rid of the child process fork/spawning. slicecamd now has 'scam autoacq'. please re-evaluate the PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants