Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modular Input Decorators #300

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

micahkemp-splunk
Copy link

In an attempt to have similar design patterns to how custom search commands are created, and also to make the definition of the scheme, member access to configuration values, automatic casting of configuration types, and convenient .spec file generation, I implemented a proof of concept for decorators for the Script class.
It mimics much of how the search command decorators are implemented, with changes where necessary.
The functionality isn't fully complete, and is lacking validation of decorator parameters, but before spending a lot more time polishing and making it complete I wanted to get feedback regarding the willingness of the SDK team to merge such a PR, and also to get early feedback regarding which pieces the team would want implemented differently.

The summary of what this PR accomplishes is shown in this code snippet:

from splunklib.modularinput import Configuration, Script, Argument
import sys

@Configuration(use_single_instance=True)
class DevSDKScript(Script):
    a_string_argument = Argument(data_type=Argument.data_type_string, description="A String Argument")
    a_boolean_argument = Argument(data_type=Argument.data_type_boolean, description="A Boolean Argument")
    a_number_argument = Argument(data_type=Argument.data_type_number, description="A Number Argument")

    # for inputs that need to perform preliminary actions (preparation) with knowledge of the entire set of inputs
    def preflight(self, input_items, ew):
        for input_name, input_item in input_items:
            ew.log("INFO", "{0} preflight".format(input_name))

    # process_input is called once per configured input to be more convenient, not requiring the developer
    #  to implement the interation themselves
    # stream_events is still overrideable if the developer prefers the existing method
    def process_input(self, input_name, input_item, ew):
        ew.log("INFO", "{0} stream_event".format(input_name))
        ew.log("INFO", " a_string_argument: {0}".format(input_item.a_string_argument))
        ew.log("INFO", " a_boolean_argument: {0}".format(input_item.a_boolean_argument))
        ew.log("INFO", " a_number_argument: {0}".format(input_item.a_number_argument))

    # for inputs that need to perform post actions (cleanup) with knowledge of the entire set of inputs
    def postflight(self, input_items, ew):
        for input_name, input_item in input_items:
            ew.log("INFO", "{0} postflight".format(input_name))

if __name__ == "__main__":
    sys.exit(DevSDKScript().run(sys.argv))

@fantavlik
Copy link
Contributor

Hi @micahkemp-splunk, right now the team is focused on addressing bug fixes and security updates so I'm not sure when an enhancement like this would be possible but could you break down what you are trying to address/make easier a bit more? This will help us determine the right priority. Also let us know if this is still something you are interested in pursuing, I know it's been nearly two years since this was put up.

For example, I think you're addressing at least four different usability problems:

  • Decorators for easier specification of configuration
  • Allowing users to define modular input arguments and validators similar to search commands such that argument parsing and validation is enforced by default by the Script class rather than requiring users to implement parsing and validation themselves
  • Providing separate methods for each stage of input gathering - preflight, process_input, and postflight
  • Enforcing that process_input is called only once per configured input: it would be great to understand how you intend to accomplish this or the issue you have with the current stream_events implementation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants