FEAT: Support for forwarding response headers from http task

[prasadlohakpure]

Description

Support for forwarding response headers from http task, and modification to jq task for non standard json record.
Why:

1. Response header: In certain scenarios, we want the capability to propagate headers down along with record data.
2. jq modification: In certain scenarios where data is non json, marshalling would return error. So if such failures pass stringified data
   (Note: Although we wont be anyway able to apply jq selector on it anyway, but this change gives capability to support selection of the complete response, and make the above JQ work.)

Use case:

1. Response header: response cookie from one response can be used in another http request to maintain session across multiple request.
2. jq modification: For jq tasks where we needed complete record selection for non json standard

Testing:

Done on test/pipelines/context_test.yaml file

Types of changes

• Docs change / refactoring / dependency upgrade
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist

• My code follows the code style of this project.
• My change requires a change to the documentation and I have updated the documentation accordingly.
• I have added tests to cover my changes.

[Copilot]

Pull request overview

This PR adds support for forwarding HTTP response headers from the HTTP task to downstream tasks via the context mechanism. Headers are automatically stored with an http-header- prefix (e.g., http-header-Content-Type) and can be accessed using context variables like {{ context "http-header-Content-Type" }}.

Changes:

• Updated placeholder regex to support hyphens in context variable names
• HTTP task now stores all response headers in record context with http-header- prefix
• Added documentation and examples for accessing response headers
• Added integration test demonstrating header access in context_test.yaml

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
internal/pkg/config/config.go	Updated placeholderRegex from ([a-zA-Z0-9_]+?) to ([a-zA-Z0-9_-]+?) to support hyphens in context keys
internal/pkg/pipeline/task/http/http.go	Added logic to store all HTTP response headers in record context with http-header- prefix
internal/pkg/pipeline/task/http/README.md	Added documentation section explaining header forwarding feature with examples
test/pipelines/context_test.yaml	Added test cases accessing http-header-Content-Type and http-header-Age headers

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[prasadlohakpure]

For certain cases where record is non json (e.g. text data, html).
JQ like this was failing:

  - name: transform_with_context_jq
    type: jq
    path: |
      {
        "headers_content_type": "{{ context "http-header-Content-Type" }}",
        "headers_cookie": "{{ context "http-header-Set-Cookie" }}",
        "original_data": .
      }

Reason : Since data is non json, Marshalling would return error.
Although we wont be anyway able to apply jq selector on it anyway, but this change gives capability to support the complete response, and make the above JQ work.