Moving? Don’t leave anything behind...

By: Rupak May 10, 2016

My client was creeping up on their massive Splunk infrastructure migration to all new hardware and they wanted a quick and simple way to be sure that the knowledge objects in their environment were migrated successfully and nothing important was left behind.

I suggested a solution that would utilize the various REST API searches that are available to gather this information and present it in a simple way.

We came up with a list of the knowledge objects and other important items they had in their environment. We ended up starting with the following six items: apps, saved searches, lookups, macros, eventtypes, and data models.

Creating the searches was a breeze with using the REST API Reference Manual as a reference.

Here is a screenshot of a dashboard that is similar to the one that was built for my client.

In the code block below, take a look at the different searches that were used to populate the dashboard above to get a better idea of the type of information and the fields that are available with the REST API.

You can drop the code directly into a new dashboard in your environment (given you have the proper access to hit the REST endpoints). Then you can update it however you please.

<dashboard>

  <label>Knowledge Object Inventory</label>

  <search id="baseSearch_apps">

    <query>| rest services/apps/local</query>

  </search>

  <search id="baseSearch_searches">

    <query>| rest services/saved/searches</query>

  </search>

  <search id="baseSearch_lookups">

    <query>| rest services/data/lookup-table-files</query>

  </search>

  <search id="baseSearch_macros">

    <query>| rest /servicesNS/-/-/admin/macros</query>

  </search>

  <search id="baseSearch_eventtypes">

    <query>| rest services/saved/eventtypes</query>

  </search>

  <search id="baseSearch_datamodels">

    <query>| rest servicesNS/admin/-/data/models</query>

  </search>

  <row>

    <panel>

      <single>

        <search base="baseSearch_apps">

          <query>stats count</query>

        </search>

        <option name="colorBy">value</option>

        <option name="colorMode">none</option>

        <option name="numberPrecision">0</option>

        <option name="showSparkline">1</option>

        <option name="showTrendIndicator">1</option>

        <option name="trendColorInterpretation">standard</option>

        <option name="trendDisplayMode">absolute</option>

        <option name="useColors">1</option>

        <option name="useThousandSeparators">1</option>

        <option name="linkView">search</option>

        <option name="drilldown">none</option>

        <option name="rangeColors">["0x6db7c6","0x6db7c6","0xd93f3c"]</option>

        <option name="rangeValues">[0,30000]</option>

        <option name="trendInterval">auto</option>

        <option name="underLabel">Apps</option>

      </single>

    </panel>

    <panel>

      <single>

        <search base="baseSearch_searches">

          <query>stats count</query>

        </search>

        <option name="colorBy">value</option>

        <option name="colorMode">none</option>

        <option name="numberPrecision">0</option>

        <option name="showSparkline">1</option>

        <option name="showTrendIndicator">1</option>

        <option name="trendColorInterpretation">standard</option>

        <option name="trendDisplayMode">absolute</option>

        <option name="useColors">1</option>

        <option name="useThousandSeparators">1</option>

        <option name="linkView">search</option>

        <option name="drilldown">none</option>

        <option name="rangeColors">["0x6db7c6","0x6db7c6","0xd93f3c"]</option>

        <option name="rangeValues">[0,30000]</option>

        <option name="trendInterval">auto</option>

        <option name="underLabel">Saved Searches</option>

      </single>

    </panel>

    <panel>

      <single>

        <search base="baseSearch_lookups">

          <query>stats count</query>

        </search>

        <option name="drilldown">none</option>

        <option name="colorBy">value</option>

        <option name="colorMode">none</option>

        <option name="numberPrecision">0</option>

        <option name="rangeColors">["0x6db7c6","0x6db7c6","0xd93f3c"]</option>

        <option name="rangeValues">[0,30000]</option>

        <option name="showSparkline">1</option>

        <option name="showTrendIndicator">1</option>

        <option name="trendColorInterpretation">standard</option>

        <option name="trendDisplayMode">absolute</option>

        <option name="trendInterval">auto</option>

        <option name="useColors">1</option>

        <option name="useThousandSeparators">1</option>

        <option name="linkView">search</option>

        <option name="underLabel">Lookups</option>

      </single>

    </panel>

    <panel>

      <single>

        <search base="baseSearch_macros">

          <query>stats count</query>

        </search>

        <option name="colorBy">value</option>

        <option name="colorMode">none</option>

        <option name="numberPrecision">0</option>

        <option name="showSparkline">1</option>

        <option name="showTrendIndicator">1</option>

        <option name="trendColorInterpretation">standard</option>

        <option name="trendDisplayMode">absolute</option>

        <option name="useColors">1</option>

        <option name="useThousandSeparators">1</option>

        <option name="linkView">search</option>

        <option name="drilldown">none</option>

        <option name="rangeColors">["0x6db7c6","0x6db7c6","0xd93f3c"]</option>

        <option name="rangeValues">[0,30000]</option>

        <option name="trendInterval">auto</option>

        <option name="underLabel">Macros</option>

      </single>

    </panel>

    <panel>

      <single>

        <search base="baseSearch_eventtypes">

          <query>stats count</query>

        </search>

        <option name="drilldown">none</option>

        <option name="colorBy">value</option>

        <option name="colorMode">none</option>

        <option name="numberPrecision">0</option>

        <option name="rangeColors">["0x6db7c6","0x6db7c6","0xd93f3c"]</option>

        <option name="rangeValues">[0,30000]</option>

        <option name="showSparkline">1</option>

        <option name="showTrendIndicator">1</option>

        <option name="trendColorInterpretation">standard</option>

        <option name="trendDisplayMode">absolute</option>

        <option name="trendInterval">auto</option>

        <option name="useColors">1</option>

        <option name="useThousandSeparators">1</option>

        <option name="linkView">search</option>

        <option name="underLabel">Event Types</option>

      </single>

    </panel>

    <panel>

      <single>

        <search base="baseSearch_datamodels">

          <query>stats count</query>

        </search>

        <option name="colorBy">value</option>

        <option name="colorMode">none</option>

        <option name="numberPrecision">0</option>

        <option name="showSparkline">1</option>

        <option name="showTrendIndicator">1</option>

        <option name="trendColorInterpretation">standard</option>

        <option name="trendDisplayMode">absolute</option>

        <option name="useColors">1</option>

        <option name="useThousandSeparators">1</option>

        <option name="linkView">search</option>

        <option name="drilldown">none</option>

        <option name="rangeColors">["0x6db7c6","0x6db7c6","0xd93f3c"]</option>

        <option name="rangeValues">[0,30000]</option>

        <option name="trendInterval">auto</option>

        <option name="underLabel">Data Models</option>

      </single>

    </panel>

  </row>

  <row>

    <panel>

      <table>

        <title>Apps</title>

        <search base="baseSearch_apps">

          <query>table title disabled eai:acl.perms.read eai:acl.perms.write eai:acl.sharing | rename title as Title disabled AS Disabled eai:acl.perms.read AS Read eai:acl.perms.write AS Write eai:acl.sharing AS Sharing</query>

          <earliest>-24h</earliest>

          <latest>now</latest>

        </search>

      </table>

    </panel>

  </row>

  <row>

    <panel>

      <table>

        <title>Saved Searches</title>

        <search base="baseSearch_searches">

          <query>table title eai:acl.app eai:acl.owner disabled is_scheduled cron_schedule next_scheduled_time dispatch.earliest_time eai:acl.perms.read eai:acl.perms.write eai:acl.sharing | rename title as Title eai:acl.app AS App eai:acl.owner AS Owner disabled AS Disabled is_scheduled AS Scheduled cron_schedule AS "Cron Schedule" next_scheduled_time AS "Next Scheduled Time" dispatch.earliest_time AS "Dispatch Earliest Time" eai:acl.perms.read AS Read eai:acl.perms.write AS Write eai:acl.sharing AS Sharing search AS Search tags AS Tags</query>

          <earliest>-24h</earliest>

          <latest>now</latest>

        </search>

      </table>

    </panel>

  </row>

  <row>

    <panel>

      <table>

        <title>Lookups</title>

        <search base="baseSearch_lookups">

          <query>table title eai:acl.app eai:acl.owner disabled updated eai:acl.perms.read eai:acl.perms.write eai:acl.sharing | rename title as Title eai:acl.app AS App eai:acl.owner AS Owner disabled AS Disabled updated AS "Last Updated" eai:acl.perms.read AS Read eai:acl.perms.write AS Write  eai:acl.sharing AS Sharing</query>

          <earliest>-24h</earliest>

          <latest>now</latest>

        </search>

        <option name="wrap">true</option>

        <option name="rowNumbers">false</option>

        <option name="dataOverlayMode">none</option>

        <option name="drilldown">cell</option>

        <option name="count">10</option>

      </table>

    </panel>

  </row>

  <row>

    <panel>

      <table>

        <title>Macros</title>

        <search base="baseSearch_macros">

          <query>table title eai:acl.app eai:acl.owner disabled definition eai:acl.perms.read eai:acl.perms.write eai:acl.sharing | rename title as Title eai:acl.app AS App eai:acl.owner AS Owner disabled AS Disabled definition AS Definition eai:acl.perms.read AS Read eai:acl.perms.write AS Write eai:acl.sharing AS Sharing</query>

          <earliest>-24h</earliest>

          <latest>now</latest>

        </search>

      </table>

    </panel>

  </row>

  <row>

    <panel>

      <table>

        <title>Eventtypes</title>

        <search base="baseSearch_eventtypes">

          <query>table title eai:acl.app eai:al.owner disabled search tags eai:acl.perms.read eai:acl.perms.write eai:acl.sharing | rename title as Title eai:acl.app AS App eai:acl.owner AS Owner disabled AS Disabled search AS Search tags AS Tags eai:acl.perms.read AS Read eai:acl.perms.write AS Write eai:acl.sharing AS Sharing</query>

          <earliest>-24h</earliest>

          <latest>now</latest>

        </search>

        <option name="wrap">true</option>

        <option name="rowNumbers">false</option>

        <option name="dataOverlayMode">none</option>

        <option name="drilldown">cell</option>

        <option name="count">10</option>

      </table>

    </panel>

  </row>

  <row>

    <panel>

      <table>

        <title>Data Models</title>

        <search base="baseSearch_datamodels">

          <query>table title eai:acl.app eai:acl.owner disabled acceleration eai:acl.perms.read eai:acl.perms.write eai:acl.sharing | replace 1 with "Enabled" 0 with "Disabled" in acceleration | rename title as Title eai:acl.app AS App eai:acl.owner AS Owner disabled AS Disabled acceleration AS Acceleration eai:acl.perms.read AS Read eai:acl.perms.write AS Write eai:acl.sharing AS Sharing</query>

          <earliest>-24h</earliest>

          <latest>now</latest>

        </search>

      </table>

    </panel>

  </row>

</dashboard>

This turned out to be extremely useful for my client as a way to compare the items from their old environment to the new environment and ensure nothing was left behind. They were also able to see a few key configuration settings for the knowledge objects that allowed them to better understand how they were being used, if they were following best practices, and if any changes needed to be made.

Review the other items and interesting fields you can include in your dashboard using the REST API Reference Manual. There is a ton of information available to you that can provide great insight into your Splunk environment.

If you need any help or have any questions, feel free to contact us at info@function1.com.

Pic courtesy of: https://blog.allstate.com/wp-content/uploads/2012/09/Moving-Scams-iStock...

Tags: Knowledge Objects, Splunk 6.3, Splunk 6.x, REST API, Lookups, macros, saved searches, event types, data models

Blog