Scraping Internal Links of a Website with Python

Following script generate a list of internal URLs of a website. To parse the HTML sources, we use BeautifulSoup and urllib libraries in Python. First list is the links that are in the main page. After that, we will follow each link to get other URLs. It goes to level 2 depth links, but you can adapt it to your needs.

This list of links can be used for a lot of reasons. I will use this list to test some security vulnerabilities.

Continue reading →

Check connection using Nmap / Linux Crontab

Sometimes you need to monitor a connection between two hosts, a single service or a host. This script check a connection using Nmap, and requires the IP and the port of the service.

Use a log file

Using a log file can be useful to generate reports, but also for checking the last status of the connection. If we don’t check last status, the script will notify for every check. We don’t want that; we want to be notified when the connection is unsuccessful and when it’s back.

Continue reading →

Jenkins with Publish over SSH

If you are not interested in Jenkins slaves, you can run remote commands and transfer files over SSH. Of course, it only works if you can connect over SSH to that remote server. You can use this practice even if you run commands on localhost, with another user.

Jenkins installation is pretty straightforward. If you manage packets with yum, you can install Jenkins with:

Continue reading →

SSH Access Limited by IP / SSH Brute Force

You can limit the SSH access by allow few IPs and deny the others. This can be done by adding in /etc/hosts.allow  file these records.

With this configuration, you allow localhost, server home network and someone’s IP. If other IP is trying to connect over SSH to this host, it will get “Connection refused”. If you look on the /var/log/secure (Ubuntu), you will see some failed connection attempts.

You can add the last line in /etc/hosts.deny . It’s the same thing.

Continue reading →

Nmap – The Good, the Bad and the Ugly

I will start with a funny comment I found on StackExchange.

Port scanning is more akin to taking a walk around the neighbourhood and trying every handle of a door and every window to see if it’s open, to see if the neighbours have left anything available for future investigation or exploitation/theft.

Anyway, nmap is an useful tool if you use it with good intentions and white-hat reasons. You can find communication channels, vulnerabilities in your network and many more.

Continue reading →

SQL Injection – Sqlmap and Google Dork

Google Dorking (or Google Hacking) is a way to find vulnerable websites and security holes in a website, searching on Google engine. Many websites expose sensitive data to the Internet. And many of them are really vulnerable to SQL Injection.

Sqlmap is an open source tool used for penetration testing. And for a lot of blackhat reasons. This software is highly sophisticated, so use it wisely (and only on your own server, or on others website, with their permission).

Continue reading →

Jenkins with SonarQube Scanner

SonarQube is an open source software used for code analysis, detecting bugs, code smells and vulnerabilities. It can be easily integrated with many CI engines. In this guide, we will see how it can be used with Jenkins.

I already have installed Jenkins 2.64 (this Jenkins version requires Java 8 in order to run). If you have Jenkins installed, you just need another Jenkins plugin.

Continue reading →