Scanning & Enumeration: Step 1. Scanning Hosts

Posted by Kulani Mahadewa on March 22, 2021
Posted in: Ethical Hacking. Tagged: , , , , , , , , , , , , . Leave a comment

Hi! 🙂 Today we are going to learn about tools on Kali Linux to perform scanning to detect potential vulnerable hosts and ports in the network. The assumption here is that the hacker has access to the network of the vulnerable host. Further, the technique we use here is an exploiting of the Address Resolution Protocol (ARP) which is used by hosts to find other hosts/devices in the same network when joining a new network.

Disclaimer: Ethical Hacking should be always performed on services that provide permissions on doing so, otherwise it cause legal issues and cost you. Generally, ethical hacking is performed to find vulnerabilities of a service by its owner or an authorized party.

Setup:

I’m using VMware Workstation Pro on my Windows machine, to run two VMware (vulnerable Linux machine and Kali Linux host). The advantage of the Pro version is that it allows to play both VMwares and interact with both at the same time. Both hosts are in the same network. The network adapter setting is setup as NAT for both VMware.

Objectives:

In Ethical Hacking, the next step after passive reconnaissance of target is scanning & enumeration. The scanning is done to detect vulnerable hosts in the network and information about those hosts. The information may include host IP, open ports, OS version, fingerprinting, and etc.

Vulnerable Host:

We are using a Kioptrix VMware as the vulnerable host. In this analysis, we know the actual vulnerable host in the network. Therefore, for the verification purpose, let’s first check what is the actual IP on vulnerable host. Since, this host do not run all the commands, we use ping to check the host IP. 

ping 8.8.8.8

Press Ctrl+ C to stop pinging. In the results we can observe that the PING is from host 192.168.5.129, which is the IP of this machine.

Scanning using Network ARP-SCAN:

ARP stands for Address Resolution Protocol. The discovery is done by broadcasting ARP packets to the hosts in the local network. Accordingly, this is what done by ARP-SCAN command to find hosts in the network.

Command: arp-scan -l or arp-scan --localnet (generate network addresses from network interface configuration)
The IP (192.168.5.129) and MAC address of the vulnerable host are detected in the results

Scanning using Netdiscover:

The netdiscover is another tool on Kali Linux that can be used for scanning the network to find live vulnerable hosts. The methodology used by this tool is also broadcasting ARP packets to the local network. The command netdiscover has option -r to specify the range of IP addresses that we are going to scan. To detect all the hosts in a network we can specify the range in CIDR notation (network ID e.g., 192.168.5.0 / subnet length).

Command: netdiscover -r 192.168.5.0/24
The IP (192.168.5.129) and MAC address of the vulnerable host are detected in the results

Now we have detected the IP address of the vulnerable host. Next, we can gather more information about the this host by using Nmap tool.

Scanning with Nmap:

With Nmap scanning, we can obtain information about the hosts in the same network such as open ports, service/version info, OS detection. The command is specified as nmap <speed – default is 4 , 5 could be too fast hence miss some info> <port range> <what information we scan><target IP>

Command: nmap -T4 -p- -A 192.168.5.129 
(-p- means scan all the ports from 0 to 65535, -A means scan everything)
Command: nmap -T4 -A 192.168.5.129 
(not specifying port range results in scanning for the top 1000 ports including all )
Command: nmap -T4 -p 80 -A 192.168.5.129 
(specify a specific port or few ports)

The results for the first command which scan for all ports and all information:

┌──(kali㉿kali)-[~]
└─$ nmap -T4 -p- -A 192.168.5.129
Starting Nmap 7.91 ( https://nmap.org ) at 2021-03-21 15:28 EDT
Nmap scan report for 192.168.5.129 (192.168.5.129)
Host is up (0.0043s latency).
Not shown: 65529 closed ports
PORT STATE SERVICE VERSION


22/tcp open ssh OpenSSH 2.9p2 (protocol 1.99)
| ssh-hostkey:
| 1024 b8:74:6c:db:fd:8b:e6:66:e9:2a:2b:df:5e:6f:64:86 (RSA1)
| 1024 8f:8e:5b:81:ed:21:ab:c1:80:e1:57:a3:3c:85:c4:71 (DSA)
|_ 1024 ed:4e:a9:4a:06:14:ff:15:14:ce:da:3a:80:db:e2:81 (RSA)
|sshv1: Server supports SSHv1

80/tcp open http Apache httpd 1.3.20 ((Unix) (Red-Hat/Linux) mod_ssl/2.8.4 OpenSSL/0.9.6b) | http-methods: | Potentially risky methods: TRACE
|http-server-header: Apache/1.3.20 (Unix) (Red-Hat/Linux) mod_ssl/2.8.4 OpenSSL/0.9.6b |_http-title: Test Page for the Apache Web Server on Red Hat Linux

111/tcp open rpcbind 2 (RPC #100000) | rpcinfo: | program version port/proto service | 100000 2 111/tcp rpcbind | 100000 2 111/udp rpcbind | 100024 1 32768/tcp status | 100024 1 32768/udp status


139/tcp open netbios-ssn Samba smbd (workgroup: MYGROUP)


443/tcp open ssl/https Apache/1.3.20 (Unix) (Red-Hat/Linux) mod_ssl/2.8.4 OpenSSL/0.9.6b
|http-server-header: Apache/1.3.20 (Unix) (Red-Hat/Linux) mod_ssl/2.8.4 OpenSSL/0.9.6b |_http-title: 400 Bad Request | ssl-cert: Subject: commonName=localhost.localdomain/organizationName=SomeOrganization/stateOrProvinceName=SomeState/countryName=– | Not valid before: 2009-09-26T09:32:06 |_Not valid after: 2010-09-26T09:32:06 |_ssl-date: 2021-03-22T08:29:50+00:00; +13h00m15s from scanner time. | sslv2: | SSLv2 supported | ciphers: | SSL2_DES_64_CBC_WITH_MD5 | SSL2_RC2_128_CBC_WITH_MD5 | SSL2_DES_192_EDE3_CBC_WITH_MD5 | SSL2_RC2_128_CBC_EXPORT40_WITH_MD5 | SSL2_RC4_128_EXPORT40_WITH_MD5 | SSL2_RC4_128_WITH_MD5 | SSL2_RC4_64_WITH_MD5


32768/tcp open status 1 (RPC #100024)

Host script results:
|_clock-skew: 13h00m14s
|_nbstat: NetBIOS name: KIOPTRIX, NetBIOS user: , NetBIOS MAC: (unknown)
|_smb2-time: Protocol negotiation failed (SMB2)

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 64.13 seconds

Based on the results we have discovered the open ports of the vulnerable machine as 22, 80, 443, 139, 111, and 32768.

Cheers ! 🙂

Tools for Passive Reconnaissance on Kali

Posted by Kulani Mahadewa on March 16, 2021
Posted in: Getting Started with Android. Leave a comment

Hi 🙂

Disclaimer: Ethical Hacking should be always performed on services that provide permissions on doing so, otherwise it cause legal issues and cost you. Generally, ethical hacking is performed to find vulnerabilities of a service by its owner or an authorized party.

To perform ethical hacking on a service that welcomes it, the first step is to gather information about the service. In particular, passive reconnaissance is the process of information gathering about a target service such as a web application. This could include personal or social information such as

  • textual data: name, email, phone number, and job information
  • image data: computer and desk at the office/home (could leak applications/tools used by a target person)

or location information (mostly physical).

Following are some tools that can be used on Kali Linux for passive reconnaissance.

  • theHarvester (built-in to Kali)

For a given domain, it searches information on sources such as google, twitter and yahoo

Command: theHarvster -d <domain> -b <source>

  • sublist3r

For a given domain (e.g., example.com) this tool finds subdomain (e.g., sub.example.com) information. It can be installed by command apt install sublist3r. Additionally, this is another online tool that can be used to perform similar information gathering.

Command: sublist3r -d <domain>

  • Wappalyzer (Browser add-on)

This tool provides meta information about the implementation of a web application such as version numbers, frameworks used, programming language used, content management used, etc.

  • Google Search

It is possible to perform effective search on Google by using specific keywords such as to find for particular type of files at a website you can use

site:<domain> filetype: <file type e.g., pdf, docx, etc>

Cheers !

Create a Simple Port Scanner on Kali Linux

Posted by Kulani Mahadewa on March 16, 2021
Posted in: Ethical Hacking. Tagged: , , , , , , . Leave a comment

Hi 🙂 This is about a simple port scanner which searches for the open ports for a given IP address. It uses python socket module. Script uses python version 3.

import socket 
import sys

start_port = 1
end_port = 65535

try:
    for port in range(start_port, end_port):
        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        socket.setdefaulttimeout(1)
        host =  socket.gethostbyname(sys.argv[1])# take IP of host as input
        result = s.connect_ex(host, port)
        if result == 0:
            print("port {} is open".format(port))
        s.close()
except Exception as ex:
    ################### check exception details ######################
    template = "An exception of type {0} occurred. Arguments:\n{1!r}"
    message = template.format(type(ex).__name__, ex.args)
    print (message)
    print('exception caught')
    sys.exit()

How to run this script:

$ python scanner.py <host IP>

Cheers !

Create Kali Linux Live USB stick with Etcher on Windows

Posted by Kulani Mahadewa on October 7, 2020
Posted in: Kali Linux. Tagged: , , , , , . Leave a comment

Hi, You can create a bootable USB stick for Kali Linux Live with few steps using the BalenaEtcher tool on Windows.

certutil -hashfile <pasth to iso image> SHA256

Once installed, you can simply select the ISO image of the kali linux live at “Flash from URL” –> Then select the USB drive for flashing at “Select target”–> Select “flash”.

Once the flashing is done, you may see as “1 failed device” due to checksum fail (during verification) on Windows. However, the writing to the USB was successful. This error has identified as a false positive case seen on Windows.

What else can you do with the Live USB stick of Kali Linux?

Add kali linux to startup on dualboot with Windows: You can find a guide to add kali linux startup on a dual boot with Windows at https://networkwolves.wordpress.com/2015/04/13/repair-kali-linux-grub-after-installing-window-in-dual-boot/amp/

Boot the live USB stick –> select “Live system” -> Open a terminal.-> login as root user

(If you have a fresh version install root user by following command)

sudo apt-get install kali-root-login
super su

Next, you can use the following commands. Then reboot the system to see the boot menu with kali linux option.

mount /dev/sda3 /mnt
mount –bind /dev /mnt/dev
mount –bind /dev/pts /mnt/dev/pts
mount –bind /proc /mnt/proc
mount –bind /sys /mnt/sys
chroot /mnt
grub-install /dev/sda
update-grub
exit
umount /mnt/dev/pts
umount /mnt/dev
umount /mnt/proc
umount /mnt/sys
umount /mnt

Resolving kali linux startup problem on dual boot desktop due to Window’s fast-restart: To resolve this also, you can use the Kali Linux Live USB stick.

When you try the first command (mount /dev/sda3 /mnt) you can see the error is shown as follows. You can disable the fast-restart which is a new feature on Windows 10, by selecting control panel->system and security -> power options–> “choose what the power buttons do” -> “Change the settings that are currently unavailable” -> untick “Turn on fast-startup”.

Cheers ! 🙂

System Execution Sequence – Part 1: BIOS/UEFI and Bootloader

Posted by Kulani Mahadewa on July 6, 2020
Posted in: System Execution Sequence. Tagged: , , , . Leave a comment

Execution Environment

Computing is about running algorithms. It is achieved by executing code. So, what is the order of executing codes in a computer?

Different kinds of computers (e.g., desktops/server platforms, mobile, and embedded devices) may organize the system execution order slightly differently. In general, on top of hardware is the kernel space, and on top of the kernel is the user-space software. However, in simple embedded software like in IoT devices, the firmware code can be on top of the hardware. The kernel provides system calls, for the user libraries to access system functionalities, and the libraries provide library calls for the applications to access the functionalities of libraries. Despite this organization, the CPU receives the data as a flat instruction stream.

Is it possible to run the binaries of different architecture on another architecture? For example, Linux binary on Windows.

Yes. In particular, there are compatibility layers that facilitate the execution of binary belongs to a different architecture. Wine is such a compatibility layer, which supports the execution of Windows binaries on Unix-like OSs. Windows Subsystem for Linux  (WSL) is another compatibility layer, which facilitates the execution of Linux native binaries on top of Windows OS.

Virtual machines (VM) are a way to provide an isolated execution environment. It can provide the same or different execution environment as the underlying architecture. Such as Linux VM provides the Linux execution environment on Windows OS. A VM provides both kernel space and userspace. Hence, the end-users can work without worrying about the underlying architecture.

Docker is another way to provide an isolated execution environment. However, docker provides isolation for application through containers, whereas VM provides isolation for the OS environment. The VM’s userspace is similar to the container of Docker.

Firmware Booting Code

When the PC is power on, firmware booting code is the first thing to run. When this booting code runs, you will see the logos on the screen. During this time, in the backend, it probes all the hardware devices, to check whether they work well. Only the keyboard is enough to communicate with this code.

BIOS (Basic I/O System) and UEFI (Unified Extensible Firmware Interface) are two types of booting codes or booting firmware. It comes pre-installed on the PC board.

BIOS vs UEFI:

  • BIOS is legacy booting firmware. It only supports up to 4 partitions of Hard Disk.
  • UEFI is a new booting firmware. It supports more than 4 partitions of HD, and additionally provide a secure mode and network support during booting to download missing codes.

Another main function of booting code is, it requires to find the next code to be executed and put it in the memory. What is the next code to execute? Bootloader.

Bootloader

It is a small program to load other OS programs and data in memory and executes them. It can be automatic or ask for manual options. With bootloader can load single or multiple kernels. It becomes the main application running on the microprocessor during normal operation.

Is it required to have a bootloader on a computer system? No. Simple embedded systems may not come with bootloader to save space.

Can you replace the existing bootloader? Yes. You can use U-boot to replace the bootloader.

Different bootloaders:

  • Linux: GRUB bootloader
  • Windows: BCD bootloader
  • Android: Fastboot (to load recovery image)
  • iOS: iboot
  • Embedded systems: Uboot (a general bootloader tool)

A main function of the bootloader is to find where the kernel image resides, and put it in the memory and execute it.

What if the bootloader cannot find the kernel or the kernel image is missing? PXE (Preboot Execution Environment) comes to the scene. An advantage of UEFI booting firmware is that it supports PXE. Hence, if the kernel image is not found, it can download it from a remote server.

The general procedure of PXE:

  • The host requests the IP for a network adapter from the DHCP server. The DHCP server replies with a temporary IP and TFTP Server address.
  • Next, the host requests a network bootloader from the TFTP server. TFTP server replies with reboot and support files.
  • Next, the host requests for kernel from the Web or TFTP server, and receives the kernel.
  • Finally, it acquires an IP for the kernel from the DHCP server.

Bootloader Security

Archived through TPM

What is TPM?

Trusted Platform Module (TPM) is a hardware component (comes on-chip of the motherboard ) that guarantees the integrity of the booting process.

Threat Model: A malicious code that replaces bootloader or kernel.

With TPM it can check the integrity of code and prevent the attacker from modifying the code at the initial stages of booting the OS. It uses simple cryptographic techniques and supports saving some sensitive data. The advantage is that it can hash a large amount of data.  The hash value is stored at PCR (Platform Configuration Register).

Chain of Trust

  • First, check whether the root of trust (BIOS/UEFI) is secure by checking the signature of the bootloader.
  • Next, transfers the control to the bootloader. At the bootloader, check the integrity of the hash value.
  • Next, transfers the control to the kernel. Check the integrity of the kernel.

UEFI Secure Mode

First, the signature of the kernel bootloader has to be generated. This is done by the manufacture. Next, when it loads check for its signature for integrity.

This adds trouble if you have to add a customized bootloader.

In fact, the UEFI secure boot + TPM is recommended for preserving security.

Unlock Bootloader

  • PC’s supports both BIOS and UEFI as booting methods. It is possible to switch between the two types. However, data loss and compatibility issues could happen.
  • In Android, the bootloader is locked by default. The fastboot tool can be used to unlock. When the bootloader is unlocked it is possible to changes the OS. (Whereas rooting will only allow the control of the whole OS. It provides access to the system partition, while bootloader unlocking provides boot or recovery partitions access).
  • In Iphones and MacOS, iboot is the bootloader.  DFU is required to downgrade iOS OS.

Booting Disk Creation and Disk Management

  • Etcher: can be used to locally generate a bootable USB with 3 steps. Select image, select drive with USE, and flash.
  • Unetbootin: Universal Netboot installer.
  • dd: direct dump, is a Unix utility. It can securely erase all files block by block.
  • Clonezilla

Online Resources that can be referred:

 TPM overview: https://www.trustedcomputinggroup.org/wp-content/uploads/TPM-Overview.pdf (optional)

Did you know that your Philips Hue bulb can be hijacked by a malicious hub in the neighborhood?

Posted by Kulani Mahadewa on May 25, 2020
Posted in: Article, Internet of Things. Tagged: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , . Leave a comment

Philips Hue is a popular smart home lighting system. This smart lighting system provides you with flexibility and ease in controlling, and diverse customizations in terms of colour range and functionalities that a typical light bulb cannot provide you. However,  there can be vulnerabilities in these new Internet of Things products due to lack of security hardening or challenges in adopting existing security solutions into them.

With reference to the previous post, HomeScan [1, 2] has discovered a vulnerability in the Philips Hue lighting system that leads to hijacking a Philips Hue bulb which is already connected with a victim’s hub at the presence of a malicious hub. As stated by HomeScan,  the vulnerability introduced due to the use of the existing communication protocol ‘ZigBee’ by the Philips Hue bulbs as a low power solution. Specifically, Philips Hue uses ZigBee Light Link (ZLL) protocol. It allows the ZigBee enabled bulb to accept and reply to the discovery beacons even after the bulb is already connected to a hub. Consequently, a malicious hub can discover the victim’s bulb by sending a discovery beacon. Following that the attacker can launch the ZLL authentication which results in the bulb disconnect itself from the victim’s hub and establish authentication with the malicious hub.

Here, is the attack demo video showing how the Philips Hue bulb is hijacked by the malicious hub over the ZigBee network. This demo uses the Perytons tool and a USB pluggable sniffing stick that supports ZigBee traffic capturing. This traffic was captured at the presence of a real Philips Hue bulb and two Philips Hue Hubs (one as the benign or belongs to the victim and the other as the malicious or belongs to the attacker). There are several other vulnerabilities that were discovered by HomeScan from including Chromecast and LIFX devices. You can check the HomeScan demo site for other findings.

 

References

[1] Scrutinizing Implementations of Smart Home Integrations. Kulani Mahadewa, Kailong Wang, Guangdong Bai, Ling Shi, Yan Liu, Jin Song Dong, and Zhenkai Liang. IEEE Transactions on Software Engineering, TSE 2019

[2] HOMESCAN: Scrutinizing Implementations of Smart Home Integrations. Kulani Mahadewa, Kailong Wang, Guangdong Bai, Ling Shi, Jin Song Dong and Zhenkai Liang. 23rd International Conference on Engineering of Complex Computer Systems, ICECCS 2018, Melbourne, Australia, December 12-14, 2018

Why Smart Homes Require Integration Analysis?

Posted by Kulani Mahadewa on May 24, 2020
Posted in: Article, Getting Started with Android. Tagged: , , , , , , , , , , , , , , , , , , , , , , , , , , , , . Leave a comment

In the HomeScan[2]  paper (in ICECCS 2018),  we highlight the importance of the security analysis of a smart home system from the integration perspective and propose a semi-automatic approach to perform the integration analysis. Later, an extended version [1] of this work was published in the 2019 TSE journal.

Why integration analysis?

Unlike the traditional ubiquitous smart home systems, the IoT (Internet of Things) driven smart home systems are more complex with the inherent attributes of IoT such as heterogeneity in technologies, standards, protocols, and platforms they are built upon while supporting a low-cost and a low-power system over the Internet.

Screenshot 2020-05-25 03.23.52

The Figure shows the diversity of communication protocols and smart devices by different manufacturers supported by a smart home system that uses Smarthings hub.

The HomeScan paper suggests two factors which make securing of a smart home system challenging as incompatibilities and invalidated assumptions exist in a smart home system due to its complexity. Therefore, it highlights the requirement of analyzing the security of a smart home system from the integration perspective to discover insecurities arrises due to the challenges in providing security for smart home systems.

How to perform integration analysis?

The paper proposes HomeScan, a semi-automatic approach to perform a security analysis of a smart home system from the integration perspective. The idea is that, if you have a model (a finite state diagram) of the integrated system, you can perform model checking against different attack models to find security problems in the integrated system. Hence, the paper first provides a set of techniques to extract the model of the integrated system as a Labeled Transition System (LTS) from the available implementations. Next, it shows that reachability checking over the generated integrated system model can be used to find security problems in the smart home integration. For more information please refer to the Homescan [2] paper or its extended version [1].

References

[1] Scrutinizing Implementations of Smart Home Integrations. Kulani Mahadewa, Kailong Wang, Guangdong Bai, Ling Shi, Yan Liu, Jin Song Dong, and Zhenkai Liang. IEEE Transactions on Software Engineering, TSE 2019

[2] HOMESCAN: Scrutinizing Implementations of Smart Home Integrations. Kulani Mahadewa, Kailong Wang, Guangdong Bai, Ling Shi, Jin Song Dong and Zhenkai Liang. 23rd International Conference on Engineering of Complex Computer Systems, ICECCS 2018, Melbourne, Australia, December 12-14, 2018

Use mitmproxy to Capture Traffic on the Same Machine

Posted by Kulani Mahadewa on April 10, 2020
Posted in: mitmproxy. Tagged: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , . Leave a comment

Hi, In this post I’m giving a detailed guide to use mitmproxy on kali Linux to capture the traffic. The mitmproxy tool provides many attacker capabilities in traffic analysis such as intercept, modify, replay, save, etc. You can check here for more details. I wanted to use this tool to capture traffic on the same machine (The tool is by default designed to use as a man-in-the-middle attacker to monitor the traffic of a victim device) to analyze the web protocols.  On my windows machine I used Fiddler to capture and analyze traffic. However, it was not enough support for Linux machines, if required to do further processing. Hence, I decided to use mitmproxy on Linux.

Note: In order to use mitmproxy to monitor traffic in the same machine we need to consider two users; one user as to provide the proxy; the other user as the victim. So following are the steps to set up your environment on a kali Linux machine. Although kali Linux comes with mitmproxy as pre-installed, I removed the existing version and installed the latest mitmproxy version 5.0.1. The newest version claims to be 4X faster.

Step 0: Remove previously installed mitmproxy

You may have to install the existing old version of mitmproxy if you are using kali Linux

sudo apt-get remove --auto-remove mitmproxy

Step 1: Create a new account for the proxy.

Use the following commands to create a new user. This user will be used as the attacker.

useradd -m mitmproxyuser// it creates a new direcotry 
 passwd mitmproxyuser //provide a password 
usermod -a -G sudo mitmproxyuser //add the user to sudo user list

Step 2: Download the mitmproxy version 5.0.1

  • Visit https://mitmproxy.org/ and download the v5.0 binary of mitmproxy.
  • Extract the downloaded .gz file. It contains following executables:
    • mitmproxy
    • mitmdump
    • mitmweb

Step 3: Install the mitmproxy as the newly created user

Change the directory to the directory with the extracted content of mitmproxy

  • Use the following command to install the mitmproxy.

Note: I’m using pip3 to install the mitmproxy, the pip gave errors for me.

sudo -u mitmproxyuser bash -c 'cd ~ && pip3 install --user mitmproxy'

If the command was successful, the following hidden files will be generated

0installmitmproxyasdiffuser(1)genfiles

Step 3: Update IP forwarding settings and add rules to the iptables.

sysctl -w net.ipv4.ip_forward=1
sysctl -w net.ipv6.conf.all.forwarding=1
sysctl -w net.ipv4.conf.all.send_redirects=0

iptables -t nat -A OUTPUT -p tcp -m owner ! --uid-owner mitmproxyuser --dport 80 -j REDIRECT --to-port 8080
iptables -t nat -A OUTPUT -p tcp -m owner ! --uid-owner mitmproxyuser --dport 443 -j REDIRECT --to-port 8080
ip6tables -t nat -A OUTPUT -p tcp -m owner ! --uid-owner mitmproxyuser --dport 80 -j REDIRECT --to-port 8080
ip6tables -t nat -A OUTPUT -p tcp -m owner ! --uid-owner mitmproxyuser --dport 443 -j REDIRECT --to-port 8080

Note*: Once the iptables are updated you will not be able to browse the internet unless the mitmproxy is started

Step 4: Start the mitmproxy as the user mitmproxyuser

sudo -u mitmproxyuser bash -c '$HOME/.local/bin/mitmproxy --mode transparent --showhost --set block_global=false'

00runmitmproxyNote*: Now, when you visit a web site on Firefox browser, it will still say that the certificate is not trusted.

01waring

Step 5: Install the mitmproxy certificates on Firefox browser

Once the mitmproxy is started …

02installcert

02details

  • Click on ‘other’ for Linux. It will download the certificate and prompt for acceptance. You can view the certificate and accept it.

02trustcert

02certdetails(2)

Step 5: Capture traffic on mitmproxy

Finally,  after installing the CA certificate, you can use the mitmproxy to analyze your traffic as follows.

03successmitmproxy

Click on an item to view the details.

03details

Cheers ! 🙂

Running PyCharm Traffic through Fiddler

Posted by Kulani Mahadewa on April 6, 2020
Posted in: Fiddler. Tagged: , , , , , , , , , , , , , , , , , , , , . Leave a comment

Hi, Here is a brief guide to setup PyCharm to send its traffic generated by requests through Fiddler.

Usage:  If you get errors while  implementing an extracted  protocol from a Fiddler traces, you can forward the traffic to Fiddler, so that you can check the captured traces against the traffic generated from PyCharm to debug your code. For example, you can solve issues when receiving Code 422 in response.

Running PyCharm Traffic through Fiddler

First, you need to update proxy settings at PyCharm with Fiddler proxy details. This will ask to install Fiddler generated certificates at PyCharm. However, this step only allows PyCharm to direct it’s traffic to Fiddler, but your python code fails since this setting does not apply to urllib3 used by requests. Hence, we have to bypass certificate verification in the code.

Step 1: Add Fiddler Generated Certificates to PyCharm

  • At PyCharm IDE Goto -> File -> Settings ->Appearance & Behaviour -> System Settings -> HTTP Proxy -> Provide Fiddler proxy settings as follows (Host: 127.0.0.1, Port: 8888)

update proxy settings

  • Restart PyCharm -> It will prompt to Accept the Fiddler generated certificates for JetBrains.

Screenshot 2020-04-06 19.32.00

  • You can check the saved certificates at File -> Settings ->Tools-> Server Certificates

Step2: Update Python Code

In the python code, I’m using requests package to send HTTPS requests. Update the request with an additional argument as verify=False. Otherwise, it will throw an verification failed exception (ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:646))

import requests

initResponse = requests.get(initURL, params=initParams, headers=headerInit,verify=False)

Now, it will only give a warning (InsecureRequestWarning: Unverified HTTPS request is being made to host ‘127.0.0.1’. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning,) and pass the code. Assuming you are aware of the security of the URL you are visiting, this can help you forward PyCharm traffic through Fiddler without much hassle.

Cheers ! 🙂

Fix ERR_CONNECTION_TIMED_OUT error on Android with Fiddler Proxy

Posted by Kulani Mahadewa on March 12, 2020
Posted in: Fiddler. Tagged: , , , , , , , , , , , , , , , , , , . Leave a comment

Hi, When setting up an Android device with Fiddler on the host machine, you need to set up proxy settings on Android Wi-Fi giving the IP address shown on Fiddler (Online) as host and port as 8888. After that, you need to visit http://ipv4.fiddler:8888 on your browser to download the Fiddler Root Certificate. However, when visiting the http://ipv4.fiddler:8888 on the Android browser, you might get ERR_CONNECTION_ TIMED_OUT.  This could be due to many reasons.

Reasons and Solutions:

  • If you are connecting your Android with Fiddler for the first time recheck the settings on Fiddler. Goto -> Tools -> Options -> Connection -> Tick “Allow remote computers to connect”.  To allow HTTPS decryption, ensure you have ticked “Decrypt HTTPS traffic” under Options-> HTTPS, and then install the root certificate.
  • If you are trying to reconnect you Android with Fiddler which was working with a different host, and now getting  ERR_CONNECTION_ TIMED_OUT on the Android browser,
    • You can first try removing current certificates on the Fiddler host machine and try reinstalling the certificates..
      • First, Goto -> Tools -> Options -> HTTPS-> Untick “Decrypt HTTPS traffic”.
      • Then, select ‘Actions’ on the same dialog -> select “Remove interception certificates”
      • Then follow the normal procedure to enable HTTPS traffic decryption
    • The ultimate try is to uninstall and reinstall the Fiddler from the beginning on your host machine. After that, your Android browser can visit http://ipv4.fiddler:8888 and download a new Fiddler Root Certificate.

Plus:

Even after successfully installing the Fiddler root certificate on your Android device, if you are not able to capture the HTTTPS traffic. It might be that you are using an older fiddler root certificate or a certificate you installed when working with a different Fiddle r host machine. Hence, remove the current installed certificate from your android device. After that, try reconnecting with the Fiddler host to reinstall the new root certificate.

adb shell
cd /data/misc/user/0/cacerts-added
ls
rm < current root certificate e.g. e5c3944b.0>

In the worst case It may be due to certificate pinning used in the application you are trying to test. In that case, you may have to use an instrumentation tool like Xposed to bypass such conditions.

Cheers ! 🙂