About Solve CTF offline disconnection problem - based on python3's small crawler

Last update: Oct 25, 2022

Related tags

Machine Learning safe_message_spider

Overview

前言

某次CTF线下比赛断网,想着搞点资料,而后参考了书鱼大佬的代码,搞了一套获取资料的小工具。

在原作者的基础上补充了安全客,freebuf,seebug完善了先知的部分代码

原作者项目地址 https://github.com/bkfish/html2markdown_Spider

参数说明

-s 需要查找的关键字，空代表按时间顺序爬取
-c 需要文章的数目，默认30
-p 是否需要建立本地图床，默认0 0代表不需要 1代表需要

安装使用

安装如下

git clone https://github.com/qiannianshuangxue/safe_message_spider
cd xianzhiSpider
pip install -r requirements.txt

先知

使用前注意设置headers的cookie

python3 xianzhiSpider.py -s CTF -c 5 -p 1

seebug

无需使用其他参数

python3 seebug_Spider.py

freebuf

无需使用其他参数

python3 freebufSpider.py

anquanke

无需使用其他参数

python3 anquankeSpider.py

查看下载的文档

由于我个人习惯使用VSCODE来查看以及搜索关键词,因此并没有开发一套查看工具。

结果

包括图床一共也只有26个G左右

效果图如下

仅供学习，禁止商用行为

Owner

天河

GitHub Repository

scikit-learn models hyperparameters tuning and feature selection, using evolutionary algorithms.

Sklearn-genetic-opt scikit-learn models hyperparameters tuning and feature selection, using evolutionary algorithms. This is meant to be an alternativ

180 Dec 20, 2022

This project has Classification and Clustering done Via kNN and K-Means respectfully

This project has Classification and Clustering done Via kNN and K-Means respectfully. It later tests its efficiency via F1/accuracy/recall/precision for kNN and Davies-Bouldin Index for Clustering. T

0 Jan 20, 2022

Neural Machine Translation (NMT) tutorial with OpenNMT-py

Neural Machine Translation (NMT) tutorial with OpenNMT-py. Data preprocessing, model training, evaluation, and deployment.

29 Jan 09, 2023

A modular active learning framework for Python

Modular Active Learning framework for Python3 Page contents Introduction Active learning from bird's-eye view modAL in action From zero to one in a fe

1.9k Dec 31, 2022

AI and Machine Learning with Kubeflow, Amazon EKS, and SageMaker

Data Science on AWS - O'Reilly Book Get the book on Amazon.com Book Outline Quick Start Workshop (4-hours) In this quick start hands-on workshop, you

2.8k Jan 03, 2023

Python package for machine learning for healthcare using a OMOP common data model

This library was developed in order to facilitate rapid prototyping in Python of predictive machine-learning models using longitudinal medical data from an OMOP CDM-standard database.

75 Jan 03, 2023

Practical Time-Series Analysis, published by Packt

Practical Time-Series Analysis This is the code repository for Practical Time-Series Analysis, published by Packt. It contains all the supporting proj

325 Dec 23, 2022

Data Version Control or DVC is an open-source tool for data science and machine learning projects

Continuous Machine Learning project integration with DVC Data Version Control or DVC is an open-source tool for data science and machine learning proj

2 Jul 29, 2021

Exemplary lightweight and ready-to-deploy machine learning project

Exemplary lightweight and ready-to-deploy machine learning project

6 Dec 20, 2022

Scikit learn library models to account for data and concept drift.

liquid_scikit_learn Scikit learn library models to account for data and concept drift. This python library focuses on solving data drift and concept d

7 Nov 18, 2021

A Pythonic framework for threat modeling

pytm: A Pythonic framework for threat modeling Introduction Traditional threat modeling too often comes late to the party, or sometimes not at all. In

644 Dec 20, 2022

Mars is a tensor-based unified framework for large-scale data computation which scales numpy, pandas, scikit-learn and Python functions.

Mars is a tensor-based unified framework for large-scale data computation which scales numpy, pandas, scikit-learn and many other libraries. Documenta

2.5k Jan 07, 2023

Applied Machine Learning for Graduate Program in Computer Science (PPGCC)

Applied Machine Learning for Graduate Program in Computer Science (PPGCC) - Federal University of Santa Catarina

1 Dec 22, 2021

Decision Weights in Prospect Theory

Decision Weights in Prospect Theory It's clear that humans are irrational, but how irrational are they? After some research into behavourial economics

32 Nov 08, 2021

Hypernets: A General Automated Machine Learning framework to simplify the development of End-to-end AutoML toolkits in specific domains.

A General Automated Machine Learning framework to simplify the development of End-to-end AutoML toolkits in specific domains.

216 Dec 23, 2022

This repo implements a Topological SLAM: Deep Visual Odometry with Long Term Place Recognition (Loop Closure Detection)

This repo implements a topological SLAM system. Deep Visual Odometry (DF-VO) and Visual Place Recognition are combined to form the topological SLAM system.

32 Jun 23, 2022

Course files for "Ocean/Atmosphere Time Series Analysis"

time-series This package contains all necessary files for the course Ocean/Atmosphere Time Series Analysis, an introduction to data and time series an

107 Nov 29, 2022

Automated Machine Learning with scikit-learn

auto-sklearn auto-sklearn is an automated machine learning toolkit and a drop-in replacement for a scikit-learn estimator. Find the documentation here

6.7k Jan 07, 2023

MooGBT is a library for Multi-objective optimization in Gradient Boosted Trees.

MooGBT is a library for Multi-objective optimization in Gradient Boosted Trees. MooGBT optimizes for multiple objectives by defining constraints on sub-objective(s) along with a primary objective. Th

66 Dec 06, 2022

Machine Learning University: Accelerated Natural Language Processing Class

Machine Learning University: Accelerated Natural Language Processing Class This repository contains slides, notebooks and datasets for the Machine Lea

2k Jan 01, 2023