Aller au contenu
login
arrow_backRetour aux issues
apache/airflow #10883

Improve ExternalTaskSensor execution date pattern

ecoDébutant verifiedApache kind:feature good first issue provider:standard

descriptionDescription

Currently in the `ExternalTaskSensor` there are two ways two set the execution date of a DAG/task the sensor is waiting on: - Provide the `execution_delta` which simply adds/subtracts a time value to/from the `execution_date` of the sensor itself - Provide the `execution_date_fn` which is a function that takes in just the `execution_date` and returns a manipulated one. I believe that the correct way to go about handling the `execution_date` for the `ExternalTaskSensor` should be analogous to the way we handle it for other operators, such as the `ExternalTaskMarker`, or more relevantly, the `TriggerDagRunOperator`, which is an operator usually used in conjunction with the `ExternalTaskSensor`. Specifically, we should remove the `execution_delta` parameter and replace it with a templated `execution_date` parameter. This not only covers the existing `execution_delta` use case through e.g. `{{ execution_date - macros.timedelta(days=1) }}`, but also provides more flexibility by enabling the benefits of templated parameters in general, such as pulling from xcoms, reading from DAG/operator params, using macros, etc. It also establishes the pattern that we see in the previously mentioned operators. The proposed solution is also sufficient enough to remove the `execution_date_fn` because we can register custom macros to manipulate the execution date the same way as an `execution_date_fn` would. But I do realize we may want to keep it for backwards compatibility reasons. First time contributor here so happy to implement/help with this to get some airflow OSS chops.
codeOuvre sur GitHub