Senior Splunk Engineer

Who You Are:

Versar, Inc., is seeking a Senior Splunk Engineer (Sr. Security Engineer) to support the DHS Enterprise Security Engineering Division within the Office of the Chief Information Officer (OCIO). The program is responsible for the architecture, design, engineering, Tier 3 operations support, maintenance, and management of the network and security infrastructure. We are seeking a Splunk Engineer to support mission-critical systems and the ideal candidate will play a key role in enhancing our security monitoring, incident detection, and response capabilities by optimizing Splunk deployments within a secure and compliant environment. You will work closely with security, IT, and compliance teams to ensure that all logging, monitoring, and reporting align with federal regulations and security frameworks. This program includes management and oversight of the technical infrastructure and security tools, sensors, servers, and data storage devices to support the DHS Network Operations Security Center (NOSC) Cyber (SOC) and other DHS stakeholders and Components.

What You'll Do:

• Design, configure, and maintain Splunk infrastructure to support security monitoring, data analytics, and operational efficiency.
• Develop and manage Splunk dashboards, alerts, reports, and visualizations that provide actionable insights for security operations and compliance auditing.
• Optimize Splunk queries (SPL) to meet agency-specific requirements for real-time monitoring, threat detection, and log analysis.
• Ensure timely onboarding of new data sources into Splunk, ensuring compliance with government data management and retention policies.
• Collaborate with internal cybersecurity, IT, and SOC teams to support threat hunting, incident response, and root-cause analysis using Splunk.
• Troubleshoot and resolve issues related to Splunk performance, indexing, and data accuracy, with a focus on maintaining compliance with government security standards (e.g., NIST, FISMA, CMMC).
• Manage Splunk upgrades, patches, and configuration changes in accordance with Change Management policies and procedures.
• Assist in maintaining Splunk environments in both on-premises and cloud-based government environments.
• Produce detailed documentation of system configurations, workflows, and processes for compliance audits and security reviews.
• Support the automation of incident detection and response processes through Splunk integrations with security orchestration tools.
• Create, manage, and support automation solutions for Splunk deployment and orchestration within a Cloud environment.
• Work closely with security engineers, other team members and application owners to solve technical problems at the network, system and application levels.
• Conduct periodic architectural reviews of installed sensors to assess effectiveness and propose optimal installation alternatives as required.
• Conduct network security architecture reviews to determine the size, and placement of intrusion monitoring equipment during the customer onboarding process.
• Documentation and Reporting along with presentation, teamwork and DHS wide collaboration are among the expected duties and mission of the task order.
• Build, implement and administer Splunk in Windows and Linux environments.
• Conceptualize, Design, Build, and Maintain current and future NOSC supported tools and platforms.

What You'll Bring:

• BA / BS in a Science, Technology, Engineering, Cybersecurity Management field
• 5 years or more of experience in Splunk engineering, administration, and data integration within government or highly regulated environments.
• Strong knowledge of Splunk SPL (Search Processing Language) to build complex queries and optimize data extraction.
• Experience with integrating, normalizing, and managing log data from various sources, including network devices, servers, cloud services, and security tools.
• Familiarity with federal security frameworks and regulations (e.g., NIST, FISMA, CMMC, FedRAMP).
• Hands-on experience with Splunk Enterprise Security (ES) and developing security use cases for monitoring and incident detection.
• Strong understanding of network security principles, operating systems (Windows, Linux), and cybersecurity tools commonly used in government environments.
• Ability to troubleshoot and resolve issues related to system performance, log ingestion, and Splunk search efficiency.
• Experience working with cross-functional teams, including IT, security operations, and compliance.
• Proficiency managing Splunk using the Splunk command-line interface and config files
• Experience onboarding data into Splunk via forwarder, scripted inputs, TCP/UDP and modular inputs from a variety of sources.
• Proficiency onboarding data using Splunk developed add-ons for Windows, Linux, and common third-party devices and applications
• Experience collaborating with separate engineering teams to configure data sources for Splunk integration
• Experience in Linux, Windows and SQL/ODBC interfaces
• Proficiency implementing and onboarding data in Splunk DB Connect
• Experience with Splunk performing systems administration, including performing installation, configuration, monitoring system performance and availability, upgrades, and troubleshooting
• Experience developing in XML, Bash, JavaScript and Python, Perl, PowerShell scripts
• General networking and security troubleshooting (firewalls, routing, NAT, etc.)
• Splunk architecture/design, implementation, and troubleshooting experience
• Experience in managing, maintaining, and administering multi-site indexer cluster
• Scripting and development skills (BASH, python, or java) with strong knowledge of regular expressions
• Proficiency developing log ingestion and aggregation strategies per Splunk best practices
• Proficiency normalizing data to Splunk Common Information Model (CIM)
• Experience implementing and optimizing Splunk data models
• Expertise developing security-focused content for Splunk, including creation of complex threat detection log and operational dashboards
• Perform integration activities to configure, connect, and pull data with 3rd party software APIs.
• Ability to autonomously prioritize and successfully deliver across a portfolio of projects
• Undertakes day-to-day operational and user support
• Knowledge of scripting languages (e.g., Python, Bash) for automating tasks and streamlining Splunk processes.
• Experience with cloud environments (e.g., AWS GovCloud, Azure Government) and their integration with Splunk.
• Experience with automation tools (e.g., Ansible, Puppet) in government infrastructure.
• Familiarity with SIEM solutions and security orchestration tools (e.g., Swimlane, Phantom, Demisto) to enhance incident response capabilities.
• Ability to meet deadlines and work independently.
• Must be proficient in English, both written and verbal, to ensure effective communication and collaboration.
• Successful results of preemployment screenings, including federal background check, MVR, and drug screen.
• Comply with company drug and alcohol policy.
• Be authorized to work in the US or will be authorized by the successful candidate's start date.

Certifications (Not Required):

• CISSP (Certified Information Security Systems Professional)
• CompTIA Security+
• ITIL Foundations
• Experience with Agile-based project management (primary Kanban)
• Splunk certifications (e.g., Splunk Core Certified Power User, Admin, Architect).
• Current Splunk Enterprise Certified Architect certification

Software/Hardware Experience Desired

• Knowledge of at least one programming or scripting language (ex. Python, PowerShell, PHP, Perl)
• Windows/Linux experience

Clearance:

• Must be able to attain/maintain DHS EOD clearance.
• Must be a US Citizen and pass a background check.
• Maintain applicable security clearance(s) at the level required by the client and/or applicable certification(s) as Requested by Softek and/or required by Softek's Client(s)/Customer(s).

Compensation

Expected Salary: $155,000 - $170,000.

Versar Global Solutions is providing the compensation range and general description of other compensation and benefits that the Company in good faith believes it might pay and/or offer for this position based on the successful applicant's education, experience, knowledge, skills, and abilities in addition to internal equity and geographic location. The Company reserves the right to ultimately pay more or less than the posted range and offer additional benefits and other compensation, depending on circumstances not related to an applicant's status protected by local, state, or federal law.

Location Requirements

Location: DHS HQ (TSA Springfield, VA), or Stennis Data Center

Telework Authorized, Remote Hybrid

Who We Are:

Headquartered in Washington, DC, Versar Global Solutions provides full mission lifecycle solutions for challenges faced by our government and commercial Customers in the natural, built, and digital environments. With nearly 2,000 team members around the world, and a rich legacy spanning more than 70 years, Versar Global Solutions delivers a broad array of planning, analysis and risk management solutions, project and program management, operations and maintenance services, and information technology applications for environmental management and remediation projects, mission critical facilities and installations, and in support of readiness and contingency operations.

EEO Commitment

Versar Global Solutions is committed to providing equal employment opportunities to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, genetic information, sexual orientation, gender identity, marital status, amnesty, or status as a covered veteran in accordance with applicable federal, state, and local laws.

Versar Global Solutions complies with applicable state and local laws governing non-discrimination in employment in every location in which the company operates. This policy applies to all terms and conditions of employment, including hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation, and training.

Versar Global Solutions expressly prohibits any form of unlawful employee harassment based on race, color, religion, sex, national origin, age, disability, genetic information, sexual orientation, gender identity, marital status, amnesty, or status as a covered veteran. Improper interference with the ability of Versar Global Solutions employees to perform their job duties is not tolerated.