SchedulerCSC¶

class lsst.ts.scheduler.scheduler_csc.SchedulerCSC(index, config_dir=None, initial_state=State.STANDBY, simulation_mode=0)¶

Bases: ConfigurableCsc

This class is a reactive component which is SAL aware and delegates work.

The SchedulerCSC is the only layer that is exposed to SAL communication. All commands recieved are by the SchedulerCSC then delegated to the responsible objects. Along with this the SchedulerCSC maintains a statemachine which is taken care of by the inherited class base_csc.BaseCsc.

Parameters:

indexint

Scheduler SAL component index. This also defines the index of the ScriptQueue the Scheduler will communicate with.

1 for the Main telescope.
2 for AuxTel.
Any allowed value (see Raises) for unit tests.

Notes

Simulation Modes

Supported simulation modes:

SchedulerModes.NORMAL: Regular operation
SchedulerModes.MOCKS3: Semi-operation mode. Normal operation but mock
s3 bucket for large file object.
SchedulerModes.SIMULATION: Simulation mode. Do not initialize driver
and other operational components.

Error Codes

300: Could not connect to the queue. This error code is published
if the Scheduler CSC can not connect to the queue.
301: Failed to put target on the queue.
400: Unspecified error on the simple target generation loop. This error
code is published if there is an unspecified error while running the simple target generation loop. For instance, if a user defined scheduling algorithm throws an exception after a call to Driver.select_next_target this error code will, most likely, be issued (along with the traceback message).
401: Unspecified error on advanced target generation loop.
500: Error updating observatory state.

Attributes:

loglogging.Log: Logging mechanism for the SchedulerCSC.
summary_statesalobj.base_csc.State: Get the summary state as a State enum.
parameters: `lsst.ts.scheduler.scheduler_csc.SchedulerCscParameters`: Object to contain parameter values to configure the SchedulerCSC.
configuration_path: `str`: Absolute path to the configuration location for the validSettings event.
configuration_repo: `str`: String of comma delimated Git repos that exist at configuration_path. ex; “origin/master,origin/develop”
valid_settings: `list` of `str`: Shortened and listed names of configuration_repo. ex; [master, develop].
models: `dict`: Dictionary of the models necessary for the SchedulerCSC to fulfill SAL commands. - location : lsst.ts.datelo.ObservatoryLocation - observatory_model : lsst.ts.observatory.model.ObservatoryModel - observatory_state : lsst.ts.observatory.model.ObservatoryState - sky : lsst.ts.astrosky.model.AstronomicalSkyModel - seeing : lsst.sims.seeingModel.SeeingModel - scheduled_downtime : lsst.sims.downtimeModel.ScheduleDowntime - unscheduled_downtime : lsst.sims.downtimeModel.UnschedulerDowntime
raw_telemetry: {`str`: :object:}: Raw, as in unparsed data that is recieved over SAL. The SchedulerCSC parses self.raw_telemtry and formats the data into self.models.
driver: `lsst.ts.scheduler.driver.Driver`: A worker class that does much of the lower level computation called by the SchedulerCSC.

Attributes Summary

`config_dir`	Get or set the configuration directory.
`default_initial_state`
`disabled_or_enabled`	Return True if the summary state is `State.DISABLED` or `State.ENABLED`.
`domain`
`enable_cmdline_state`
`need_to_generate_target_queue`	Check if we need to generate target queue.
`simulation_help`
`simulation_mode`	Get the current simulation mode.
`summary_state`	Get the summary state as a `State` enum.
`valid_simulation_modes`
`version`

Methods Summary

`add_arguments`(parser)	Add arguments to the parser created by `make_from_cmd_line`.
`add_kwargs_from_args`(args, kwargs)	Add constructor keyword arguments based on parsed arguments.
`advance_target_production_loop`()	Advance target production loop.
`amain`(index, **kwargs)	Make a CSC from command-line arguments and run it.
`assert_enabled`([action])	Assert that an action that requires ENABLED state can be run.
`assert_idle`()	Assert detailed state is idle.
`assert_running`()	Assert detailed state is running.
`begin_disable`(data)	Transition from `State.ENABLED` to `State.DISABLED`.
`begin_enable`(data)	Begin do_enable.
`begin_exitControl`(data)	Begin do_exitControl; called before state changes.
`begin_standby`(data)	Begin do_standby; called before the state changes.
`begin_start`(data)	Begin do_start; configure the CSC before changing state.
`check_scheduled_targets`()	Loop through the scheduled targets list, check status and tell driver of completed observations.
`check_targets_queue_condition`()	Check targets queue condition.
`close`([exception, cancel_start])	Shut down, clean up resources and set done_task done.
`close_tasks`()	Shut down pending tasks.
`compute_predicted_schedule`(args, *kwargs)
`configure`(config)	This method is responsible for configuring the scheduler models and the scheduler algorithm, given the input setting.
`current_scheduler_state`(publish_lfoa)	A context manager to handle storing the current scheduler state, performing some operations on it and then resetting it to the previous state.
`detailed_state`(detailed_state)	Context manager to set the detailed state for an operation then return it to the initial value after it executes.
`do_disable`(data)	Transition from `State.ENABLED` to `State.DISABLED`.
`do_enable`(data)	Transition from `State.DISABLED` to `State.ENABLED`.
`do_exitControl`(data)	Transition from `State.STANDBY` to `State.OFFLINE` and quit.
`do_load`(data)	Load user-defined schedule definition from URI.
`do_resume`(data)	Resume target production loop.
`do_setAuthList`(data)	Update the authorization list.
`do_setLogLevel`(data)	Set logging level.
`do_standby`(data)	Transition from `State.DISABLED` or `State.FAULT` to `State.STANDBY`.
`do_start`(data)	Transition from `State.STANDBY` to `State.DISABLED`.
`do_stop`(data)	Stop target production loop.
`end_disable`(data)	End do_disable; called after state changes but before command acknowledged.
`end_enable`(data)	End do_enable; called after state changes but before command acknowledged.
`end_exitControl`(data)	End do_exitControl; called after state changes but before command acknowledged.
`end_standby`(data)	End do_standby; called after state changes but before command acknowledged.
`end_start`(data)	End do_start; called after state changes but before command acknowledged.
`estimate_next_target`()	Estimate how long until the next target become available.
`fault`(code, report[, traceback])	Enter the fault state and output the `errorCode` event.
`generate_target_queue`(args, *kwargs)
`get_config_pkg`()	Get the name of the configuration package, e.g.
`get_queue`([request])	Utility method to get the queue.
`handle_no_targets_on_queue`()	Handle condition when there are no more targets on the queue.
`handle_observatory_state`()	Handle observatory state.
`handle_summary_state`()	Handle summary state.
`implement_simulation_mode`(simulation_mode)	Implement going into or out of simulation mode.
`make_from_cmd_line`(index, **kwargs)	Construct a CSC from command line arguments.
`put_log_level`()	Output the logLevel event.
`put_on_queue`(targets)	Given a list of targets, append them on the queue to be observed.
`queue_targets`(args, *kwargs)
`read_config_dir`()	Read the config dir and put configurationsAvailable if changed.
`read_config_dir_loop`()
`read_config_files`(config_validator, ...[, ...])	Read a set of configuration files and return the validated config.
`register_observation`(target)	Register observation.
`remove_from_queue`(targets)	Given a list of targets, remove them from the queue.
`reset_handle_no_targets_on_queue`()	Reset conditions that no targets on queue were handled.
`save_scheduler_state`(publish_lfoa)	Save scheduler state to S3 bucket and publish event.
`set_simulation_mode`(simulation_mode)	Set the simulation mode.
`simple_target_production_loop`()	This coroutine implements the simple target production loop.
`start`()	Finish constructing the CSC.
`start_phase2`()	Handle the initial state.
`stop_next_target_timer_task`()
`telemetry_loop`()	Scheduler telemetry loop.

Attributes Documentation

config_dir¶

Get or set the configuration directory.

Parameters:

config_dirstr, pathlib.Path: New configuration directory.

Returns:

config_dirpathlib.Path: Absolute path to the configuration directory.

Raises:

ValueError: If the new configuration dir is not a directory.

default_initial_state: State = 5¶

disabled_or_enabled¶

Return True if the summary state is State.DISABLED or State.ENABLED.

This is useful in handle_summary_state to determine if you should start or stop a telemetry loop, and connect to or disconnect from an external controller

domain¶

enable_cmdline_state = False¶

need_to_generate_target_queue¶

Check if we need to generate target queue.

The condition in which we have to generate a target queue is when the targets_queue_condition future is done and its result is None. If the future is not done, the task that checks the queue is still ongoing. If the results is different than None it means the queue is ok and it does not need to be generated.

Note that this is also the initial condition, so the target list is generated the first time the loop runs.

Returns:

bool: True if we need to call generate target queue, False otherwise.

simulation_help: None | str = None¶

simulation_mode¶

Get the current simulation mode.

0 means normal operation (no simulation).

Raises:

ExpectedError: If the new simulation mode is not a supported value.

summary_state¶: Get the summary state as a State enum.

valid_simulation_modes: Sequence[int] = (0, 1, 2)¶

version = '1.19.2.dev13+g7e6e3ec'¶

Methods Documentation

classmethod add_arguments(parser: ArgumentParser) → None¶

Add arguments to the parser created by make_from_cmd_line.

Parameters:

parserargparse.ArgumentParser: The argument parser.

Notes

If you override this method then you should almost certainly override add_kwargs_from_args as well.

classmethod add_kwargs_from_args(args: Namespace, kwargs: dict[str, Any]) → None¶

Add constructor keyword arguments based on parsed arguments.

Parameters:

argsargparse.Namespace: Parsed command.
kwargsdict: Keyword argument dict for the constructor. Update this based on args. The index argument will already be present if relevant.

Notes

If you override this method then you should almost certainly override add_arguments as well.

async advance_target_production_loop()¶

Advance target production loop.

This method will schedule targets ahead of time and send them to the ScriptQueue according to the configuration parameters. This initial implementation is simply blindly scheduling targets ahead of time and not checking for post conditions. Further improvements will involve dealing with different post conditions, generating the predicted queue and further verifying that the schedule is keeping up.

async classmethod amain(index: int | IntEnum | bool | None, **kwargs: Any) → None¶

Make a CSC from command-line arguments and run it.

Parameters:

indexint, enum.IntEnum, True, or None

If the CSC is indexed, do one of the following:

Specify True to make index a required command-line argument that accepts any nonzero index.
Specify an enum.IntEnum class to make index a required command-line argument that only accepts the enum values.
Specify a non-zero integer to use that index. This is rare; if the CSC is indexed then the user should usually be allowed to specify the index.

If the CSC is not indexed, specify None or 0.

**kwargsdict, optional

Additional keyword arguments for your CSC’s constructor.

assert_enabled(action: str = '') → None¶

Assert that an action that requires ENABLED state can be run.

Parameters:

actionstr, optional: Action attempted. Not needed if this is called at the beginning of a do_... method, since the user will know what command was called.

assert_idle()¶: Assert detailed state is idle.

assert_running()¶: Assert detailed state is running.

async begin_disable(data)¶

Transition from State.ENABLED to State.DISABLED. This transition will be made in a gentle way, meaning that it will wait for the target production loop to finalize before making the transition. If a more drastic approach is need, the scheduler must be sent to State.FAULT with an abort command.

Parameters:

dataDataType: Command data

async begin_enable(data)¶

Begin do_enable.

Called before state transition from State.DISABLED to State.ENABLED. This method will start the selected target production loop (SIMPLE, ADVANCED or DRY) or raise an exception if the mode is unrecognized.

Parameters:

dataDataType: Command data

async begin_exitControl(data: BaseMsgType) → None¶

Begin do_exitControl; called before state changes.

Parameters:

dataDataType: Command data

async begin_standby(data: BaseMsgType) → None¶

Begin do_standby; called before the state changes.

Parameters:

dataDataType: Command data

async begin_start(data)¶

Begin do_start; configure the CSC before changing state.

Parameters:

datacmd_start.DataType: Command data

Notes

The override field must be one of:

The name of a config label or config file
The name and version of a config file, formatted as <file_name>:<version>, where the version is a git reference, such as a git tag or commit hash. This form does not support labels.

async check_scheduled_targets()¶

Loop through the scheduled targets list, check status and tell driver of completed observations.

Returns:

bool: True if all checked scripts where Done or in non-final state. False if no scheduled targets to check or if one or more scripts ended up a failed or unrecognized state.

async check_targets_queue_condition()¶

Check targets queue condition.

For now the check is only verifying if more targets need to be added to the internal queue. In the future this method will be responsible for verifying that the predicted schedule is still fine and requesting a re-scheduling if needed.

async close(exception: None | Exception = None, cancel_start: bool = True) → None¶

Shut down, clean up resources and set done_task done.

May be called multiple times. The first call closes the Controller; subsequent calls wait until the Controller is closed.

Subclasses should override close_tasks instead of close, unless you have a good reason to do otherwise.

Parameters:

exceptionException, optional: The exception that caused stopping, if any, in which case the self.done_task exception is set to this value. Specify None for a normal exit, in which case the self.done_task result is set to None.
cancel_startbool, optional: Cancel the start task? Leave this true unless calling this from the start task.

Notes

Removes the SAL log handler, calls close_tasks to stop all background tasks, pauses briefly to allow final SAL messages to be sent, then closes the dds domain.

async close_tasks() → None¶: Shut down pending tasks. Called by close.

async compute_predicted_schedule(*args, **kwargs)¶

async configure(config)¶

This method is responsible for configuring the scheduler models and the scheduler algorithm, given the input setting. It will raise an exception if the input setting is not valid.

Parameters:

configtypes.SimpleNamespace: Configuration, as described by schema/Scheduler.yaml

Returns:

None

current_scheduler_state(publish_lfoa)¶

A context manager to handle storing the current scheduler state, performing some operations on it and then resetting it to the previous state.

Parameters:

publish_lfoabool: Publish current state to large file annex?

detailed_state(detailed_state)¶

Context manager to set the detailed state for an operation then return it to the initial value after it executes.

This method will acquire a lock to prevent executing a detailed state operation inside another.

Parameters:

detailed_stateDetailedState: Detailed state value.

async do_disable(data: BaseMsgType) → None¶

Transition from State.ENABLED to State.DISABLED.

Parameters:

datacmd_disable.DataType: Command data

async do_enable(data: BaseMsgType) → None¶

Transition from State.DISABLED to State.ENABLED.

Parameters:

datacmd_enable.DataType: Command data

async do_exitControl(data: BaseMsgType) → None¶

Transition from State.STANDBY to State.OFFLINE and quit.

Parameters:

datacmd_exitControl.DataType: Command data

async do_load(data)¶

Load user-defined schedule definition from URI.

The file is passed on to the driver to load. It must be compatible with the currently configured scheduling algorithm or the load will fail, and the command will be rejected.

Parameters:

dataDataType: Command data

async do_resume(data)¶

Resume target production loop.

Parameters:

dataDataType: Command data

Raises:

RuntimeError: If target production loop is already running.

async do_setAuthList(data: BaseMsgType) → None¶

Update the authorization list.

Parameters:

datacmd_setAuthList.DataType: Authorization lists.

Notes

Add items if the data string starts with “+”, ignoring duplicates (both with respect to the existing items and within the data string). Remove items if the data string starts with “-”, ignoring missing items (items specified for removal that do not exist). Ignore whitespace after each comma and after the +/- prefix.

async do_setLogLevel(data: BaseMsgType) → None¶

Set logging level.

Parameters:

datacmd_setLogLevel.DataType: Logging level.

async do_standby(data: BaseMsgType) → None¶

Transition from State.DISABLED or State.FAULT to State.STANDBY.

Parameters:

datacmd_standby.DataType: Command data

async do_start(data: BaseMsgType) → None¶

Transition from State.STANDBY to State.DISABLED.

Parameters:

datacmd_start.DataType: Command data

async do_stop(data)¶

Stop target production loop.

Parameters:

dataDataType: Command data

async end_disable(data: BaseMsgType) → None¶

End do_disable; called after state changes but before command acknowledged.

Parameters:

dataDataType: Command data

async end_enable(data: BaseMsgType) → None¶

End do_enable; called after state changes but before command acknowledged.

Parameters:

dataDataType: Command data

async end_exitControl(data: BaseMsgType) → None¶

End do_exitControl; called after state changes but before command acknowledged.

Parameters:

dataDataType: Command data

async end_standby(data: BaseMsgType) → None¶

End do_standby; called after state changes but before command acknowledged.

Parameters:

dataDataType: Command data

async end_start(data: BaseMsgType) → None¶

End do_start; called after state changes but before command acknowledged.

Parameters:

dataDataType: Command data

async estimate_next_target()¶: Estimate how long until the next target become available.

async fault(code: None | int, report: str, traceback: str = '') → None¶

Enter the fault state and output the errorCode event.

Parameters:

codeint: Error code for the errorCode event. If None then errorCode is not output and you should output it yourself. Specifying None is deprecated; please always specify an integer error code.
reportstr: Description of the error.
tracebackstr, optional: Description of the traceback, if any.

async generate_target_queue(*args, **kwargs)¶

static get_config_pkg()¶: Get the name of the configuration package, e.g. “ts_config_ocs”.

async get_queue(request: bool = True) → BaseMsgType¶

Utility method to get the queue.

Parameters:

requestbool: Issue request for queue state?

Returns:

queue: ScriptQueue_logevent_queueC: SAL Topic with information about the queue.

async handle_no_targets_on_queue()¶: Handle condition when there are no more targets on the queue.

async handle_observatory_state()¶: Handle observatory state.

async handle_summary_state()¶

Handle summary state.

If the component is DISABLED or ENABLED, it will make sure the telemetry loop is running. Shutdown the telemetry loop if in STANDBY.

async implement_simulation_mode(simulation_mode: int) → None¶

Implement going into or out of simulation mode.

Deprecated. See simulation mode for details.

Parameters:

simulation_modeint: Requested simulation mode; 0 for normal operation.

Raises:

ExpectedError: If simulation_mode is not a supported value.

classmethod make_from_cmd_line(index: int | IntEnum | bool | None, **kwargs: Any) → BaseCsc¶

Construct a CSC from command line arguments.

Parameters:

indexint, enum.IntEnum, True, or None

If the CSC is indexed, do one of the following:

Specify True to make index a required command-line argument that accepts any nonzero index.
Specify an enum.IntEnum class to make index a required command-line argument that only accepts the enum values.
Specify a non-zero integer to use that index. This is rare; if the CSC is indexed then the user should usually be allowed to specify the index.

If the CSC is not indexed, specify None or 0.

**kwargsdict, optional

Additional keyword arguments for your CSC’s constructor.

Returns:

csccls: The CSC.

Notes

To add additional command-line arguments, override add_arguments and add_kwargs_from_args.

async put_log_level() → None¶: Output the logLevel event.

async put_on_queue(targets)¶

Given a list of targets, append them on the queue to be observed. Each target sal_index attribute is updated with the unique identifier value (salIndex) returned by the queue.

Parameters:

targetslist: A list of targets to put on the queue.

async queue_targets(*args, **kwargs)¶

async read_config_dir() → None¶

Read the config dir and put configurationsAvailable if changed.

Output the configurationsAvailable event (if changed), after updating the overrides and version fields. Also update the version field of evt_configurationApplied, in preparation for the next time the event is output.

async read_config_dir_loop() → None¶

classmethod read_config_files(config_validator: Draft7Validator, config_dir: Path, files_to_read: list[str], git_hash: str = '') → SimpleNamespace¶

Read a set of configuration files and return the validated config.

Parameters:

config_validatorjsonschema validator: Schema validator for configuration.
config_dirpathlib.Path: Path to config files.
files_to_readList [str]: Names of files to read, with .yaml suffix. Empty names are ignored (a useful feature for BaseConfigTestCase). The files are read in order, with each later file overriding values that have been accumulated so far.
git_hashstr, optional: Git hash to use for the files. “” if current.

Returns:

types.SimpleNamespace: The validated config as a simple namespace.

Raises:

ExpectedError: If the specified configuration files cannot be found, cannot be parsed as yaml dicts, or produce an invalid configuration (one that does not match the schema).

async register_observation(target: DriverTarget) → None¶

Register observation.

Parameters:

observationObservation: Observation to be registered.

async remove_from_queue(targets)¶

Given a list of targets, remove them from the queue.

Parameters:

targets: `list`: A list of targets to put on the queue.

async reset_handle_no_targets_on_queue()¶: Reset conditions that no targets on queue were handled.

async save_scheduler_state(publish_lfoa)¶

Save scheduler state to S3 bucket and publish event.

Parameters:

publish_lfoabool: Publish current state to large file annex?

Returns:

str: Path to the current scheduler state snapshot.

async set_simulation_mode(simulation_mode: int) → None¶

Set the simulation mode.

Await implement_simulation_mode, update the simulation mode property and report the new value.

Parameters:

simulation_modeint: Requested simulation mode; 0 for normal operation.

async simple_target_production_loop()¶

This coroutine implements the simple target production loop. It will query the status of the queue and, if there is nothing running, it will add an observation to the back of the queue. Once Scheduler is enabled with simple mode, this coroutine will be added to the event loop as a future. While running it will checking the queue and put targets to it. If the queue is paused or there is a running script, it will not add any target to it. If the Scheduler is disabled this method will be cancelled.

Also, note that the target production loop does not care about if it is day or night. It is up to the scheduling algorithm to decide whether there is something to observe or not given the input telemetry. A target produced by the scheduler can have a start time. In this case, the task will be sent to the queue and it is up to the queue or the running script to wait for the specified time. If the scheduling algorithm does not produce any target, then the Scheduler will keep updating telemetry and requesting targets until something is returned to be executed.

NOTE: This method may either run in the main event loop or in an event loop of a thread. If it is the first case, we need to make sure long running tasks are sent to a thread or a process. If the latter, then it should be ok to simple execute long running tasks and block the thread event loop.

async start() → None¶

Finish constructing the CSC.

Call set_simulation_mode. If this fails, set self.start_task to the exception, call stop, making the CSC unusable, and return.
Call handle_summary_state
Set self.start_task done.

async start_phase2() → None¶

Handle the initial state.

Called after start.

async stop_next_target_timer_task()¶

async telemetry_loop()¶

Scheduler telemetry loop.

This method will monitor and process the observatory state and publish the information to SAL.

Navigation

SchedulerCSC¶