I’m somewhat new to working with APIs and scom, but I’ve been trying to use the Web API to pull some data from splunk into scom. I’ve been using the below guide which is good but I have a few questions I’m hoping someone can help me with.
Why is my splunk search a POST instead of a GET according to the documentation? I'm "getting" information from splunk, not posting any data to it.
Why when i do output_mode to json, it doesn't auto-generate the column names? Without these, I get [object],[object], etc which says to me these would be columns, but when I add the json piece, it gives me an error that splunk returned unexpected data
Any common gotchas trying to do this kind of API call with splunk? Right now I'm running what should be a basic search
There’s a couple of things in the mix here, but the overall point is that you’re not getting search results, you’re asking for a search to be carried out. The WebAPI tile is actually making the search happen, and through one of the entries in the form data, its asking Splunk to include the search results in the response body.
GET search/jobs: by defualt will return a list of recent job IDs. If you specify a job ID, it’ll return the results of that job. In this case, the search needs to have already been carried out, and you need to know the ID for the job in question. Every job has a unique ID so if you GET against this endpoint and specify a fixed job ID, the job’s search results are always historic i.e. from when that job originally ran.
Because you’re probably looking for “live” data, you dont just want to GET some results from an old job, you want to POST a request to run a brand new search whenever the dashboard loads and get its results on-demand. This is where POST comes in.
POST search/jobs: initiates a search based on the parameters specified in the form data In the headers & data section of the tile config, the search parameter is the search you’re running (in Splunk’s native language). output_mode forces the response to be JSON formatted, which SquaredUp requires, and the “magic” comes from the exec_mode parameter. By default, the response to a POST against this endpoint is just the ID of the newly created job. By using exec_mode:oneshot, Splunk will instead run the search and return the results in “one shot”
One other vital thing is to specify the right key path in the response data section of the tile (step 11). Not doing this is where you’ll see the issue you mentioned in question 2. The response from Splunk is structured in a way that requires you to tell SquaredUp where to start reading from. Basically you’re saying to ignore the top line of the response and instead start reading further down the structure. When you drop down to the results key path, SquaredUp will then be able to find all the columns that contain your actual data.
If you are having trouble I’d recommend just deleting your WebAPI tile, create a new one, and follow the guide to the end. Dont worry about the results you see until you’ve got at least past step 11 where you’ve specified the key path. At that point, you should see a table with a load of columns.