Senior Software Engineer - Crawler Team at Siteimprove

Posted on: 07/14/2021

Location: København (ON-SITE)

full time

Crunchbase | Original Source

Tags: harbor redis oracle golang kubernetes kafka python aws

**Crawler Team - Senior Software Engineer** Imagine if all the Internet was accessible to all people (incl. people with disabilities), searches always returned exactly what you were looking for, pages loaded quickly, web sites can be trusted to be secure and the quality of the content was just… better? We’re building the tools that help companies around the world achieve just that. **The opportunity** We crawl the web sites of our customers in order to provide detailed quality, accessibility, SEO and security analysis and help our customers excel in their web presence and overall improve the Web. All this is built upon our crawler technologies, processing massive amounts of web pages every day. We’re looking for experienced back-end engineers to expand our Crawler team, and join us in enhancing our core capabilities while we’re substantially scaling the business. Although we are an onsite-first company, we’re very flexible and open to colleagues working remotely part of the week. **Why is now a really exciting time to join?** We have recently completed a re-architecture of our Crawler stack, and the code base is as clean as ever. This has given us the opportunity to bring completely new features to our customers, that we believe will revolutionize our product and our business. We’re growing the team to make sure we can deliver on those promises, and expect our work to be a major factor in the continued growth of our company. **What you’ll be doing** In the Crawler team you will be working on further enhancing the robustness, scalability and correctness of the Crawler, which generally works like the crawler of any search-provider. You will be working on optimizing the downloading of resources (fine-tuning HTTP controls, caching layers), improving our HTML parsing algorithm to handle all the strange things the Internet throws at us, coordinating the many tasks that crawling entails across a complex distributed system and providing users with metrics and insights into their crawls. We’re a collaboration-focused team, so you will be working closely together with the rest of your colleagues on this. **Who we’re looking for** Ideally you’re an experienced engineer who has worked with distributed systems in the past. We are very interested in hearing about the products you have been part of delivering, be they successes or failures (we call them learnings). You should have an open mindset and both be open to feedback and willing to give feedback to your colleagues. Our technology stack includes: * GoLang, Python * Kubernetes * Kafka, Redis, various AWS services * Oracle We strive to have people with different specialities on the team, and it’s difficult to predict exactly what will help us, so don’t be afraid to write us if you have a specialty you feel would bring value to our team but isn’t necessarily mentioned here. Also, we have good experience with new colleagues quickly getting up to speed with the tech we use, so don’t worry if you can’t check all the boxes. **Life at Siteimprove** Our Product Department is organized as cross-functional product teams with a high degree of autonomy, and a few platform teams which ties it all together. The Crawler team is somewhere in the middle of that spectrum, as we are responsible for an important customer-facing component, and also deliver raw data to the downstream product teams. We are proud of our engineering culture and community spirit. We host weekly internal tech talks, public meetups for various user groups and have recently started our own [Youtube tech talk channel]( "Youtube tech talk channel"): <> where we will be curating talks by both internal and external speakers. Furthermore, we have an engineering forum where everyone can chime in on architectural decisions, submit proposals for changes or new things and solicit or provide feedback. We want to be fully transparent about our engineering, and encourage everyone to help in moving us forward. In Siteimprove you will be able to grow your technical career as far as you like. You will be surrounded by technologists with many different interests and backgrounds. We will support you in both learning new skills, growing your responsibilities and coaching your fellow engineers. Personal excellence and collaboration are the cornerstones of how we work. **Benefits** * Hybrid workplace, work from home or from the office or a mix * Amazing colleagues with a great mix of cultures * Excellent lunch (we have our own kitchen staff and we love them!) * Amazing office facilities right in the middle of Copenhagen with a view of the harbor * Lots of social activities (Friday bars, beer brewing, sports, wine tasting, board games etc.) * In-house gym with external trainer course + shower facilities * In-house brewery and bar **How To Apply** Click on the ‘Apply Now’ button to submit your application. If you have any questions regarding the job or need a reasonable accommodation to apply, feel free to contact our HR Business Partner, Jacob Bjerrehuus at []( Siteimprove is a global corporation and has developed data practices designed to assure your personally-identifiable information is appropriately protected. Please note that personal information may be transferred, accessed and stored globally as necessary for the uses and disclosures stated in accordance with our Privacy Policy at