Scraping Internal Links of a Website with Python

Following script generate a list of internal URLs of a website. To parse the HTML sources, we use BeautifulSoup and urllibĀ libraries in Python. First list is the links that are in the main page. After that, we will follow each link to get other URLs. It goes to level 2 depth links, but you can adapt it to your needs.

This list of links can be used for a lot of reasons. I will use this list to test some security vulnerabilities.

Continue reading →

Check connection using Nmap / Linux Crontab

Sometimes you need to monitor a connection between two hosts, a single service or a host. This script check a connection using Nmap, and requires the IP and the port of the service.

Use a log file

Using a log file can be useful to generate reports, but also for checking the last status of the connection. If we don’t check last status, the script will notify for every check. We don’t want that; we want to be notified when the connection is unsuccessful and when it’s back.

Continue reading →

Jenkins with Publish over SSH

If you are not interested in Jenkins slaves, you can run remote commands and transfer files over SSH. Of course, it only works if you can connect over SSH to that remote server. You can use this practice even if you run commands on localhost, with another user.

Jenkins installation is pretty straightforward. If you manage packets with yum, you can install Jenkins with:

Continue reading →

Jenkins with SonarQube Scanner

SonarQube is an open source software used for code analysis, detecting bugs, code smells and vulnerabilities. It can be easily integrated with many CI engines. In this guide, we will see how it can be used with Jenkins.

I already have installed Jenkins 2.64 (this Jenkins version requires Java 8 in order to run). If you have Jenkins installed, you just need another Jenkins plugin.

Continue reading →