Clik here to view.
Python实现并行抓取整站40万条房价数据（可更换抓取城市）
写在前面这次的爬虫是关于房价信息的抓取，目的在于练习10万以上的数据处理及整站式抓取。数据量的提升最直观的感觉便是对函数逻辑要求的提高，针对python的特性，谨慎的选择数据结构。以往小数据量的抓取，即使函数逻辑部分重复，I/O请求频率密集，循环套嵌过深，也不过是1~2s的差别，而随着数据规模的提高，这1~2s的差别就有可能扩展成为1~2h。因此对于要抓取数据量较多的网站，可以从两方面着手降低抓取信...
View Article从零开始学Python第八周：详解网络编程基础（socket)
一，Socket编程（1）Socket方法介绍 Socket是网络编程的一个抽象概念。通常我们用一个Socket表示“打开了一个网络链接“，而打开一个Socket需要知道目标计算机的IP地址和端口号，再指定协议类型即可。 套接字是一个双向的通信信道的端点。套接字可能在沟通过程，进程之间在同一台机器上，或在不同的计算机之间的进程...
View ArticlePython 'takes exactly 1 argument (2 given)' Python error
python初学，定义urlConfig 接收参数，正常传递参数时，出现，多给了一个参数的错误问题，定义class的函数之后，在调用的时候出现“'takes exactly 1 argument (2 given)' Python error”。查询Interesting ‘takes exactly 1 argument (2 given)' Python error...
View ArticlePython中的元组(tuple)、列表(list)、字典(dict)
-------------------------------更新中-------------------------------------- 元组(tuple)： 元组常用小括号表示，即：()，元素加逗号，是元组的标识。 1 #定义一个元组 2 3 #tuple = 'a', 4 5 tuple = ('a','b','c','d','e','f','g') 6 7...
View ArticleClik here to view.
Project A Part 3: From July toOctober
This post is part of a series. Part two ishere. I had meant to keep a seriously thorough engineering log of my work on Project A, in order to write some nice, spaced out blog posts about it. Instead,...
View ArticleClik here to view.
Writing Burp Extensions (Shodan Scanner)
In this article, we will have an overview of writing Burp extensions. At the end of the post, we will have an extension that will take any HTTP request, determine the IP address of domain and get...
View ArticleData manipulation with numpy: tips and tricks, part 1
Data manipulation with numpy: tips and tricks, part 1 Some inobvious examples of what you can do with numpy are collected here. Examples are mostly coming from area of machine learning, but will be...
View ArticleLintel Technologies: txRedis
txredis is a non-blocking client for the redis database, written in python. It uses twisted for the asynchronous communication with redis. Install pipinstalltxredis Now to check if txredis is properly...
View ArticleClik here to view.
How to Identify and Remove Seasonality from Time Series Data with Python
Time series datasets can contain a seasonal component. This is a cycle that repeats over time, such as monthly or yearly. This repeating cycle may obscure the signal that we wish to model when...
View Article如何创建自己的ansible module
原文地址： http://blog.toast38coza.me/custom-ansible-module-hello-world/ 什么是 ansible module? Ansible modules 是一种使用python写成的一些功能块，可以在yaml文件中调用，也就是playbook中可以调用的模块，例如常见的模块 copy ， debug Ansible 本身就提供了很多很多的模块...
View ArticleThe Python Tutorial Reading Notes
The python Tutorial is a good stuff to learn Python. I have already read it sketchily before, but didn't foucs on some detail things. This time I read it again and write some notes for reference next...
View ArticleGlobal logging with flask
As of December 2016, Flask has a built-in logger that it instantiates for you. Unfortunately, this misses the errors and other log messages in other libraries that may also be valuable. It would be...
View ArticleThis is fine
The BBC micro:bit is a tool for introducing young people to programming. It’s a little embedded computer with a few inputs and a matrix of LEDs for output, as well as some control lines. In principle...
View ArticleKrita Devs Work on SVG Support, Python Scripting, and Text Tools for Krita 3.2
Now thatKrita 3.1 launched as the latest stable release of the 3.x series, the development team of the popular, open-source and cross-platform digital painting software shared with us some of the...
View ArticleVasudev Ram: Using psutil to get disk partition information with Python
By Vasudev Ram psutil is a python library that enables you to get various kinds of operating system information, such as about CPUs , processes , memory and disks . Here is a short program that shows...
View ArticleClik here to view.
通过Python抓取天猫评论数据
天气逐渐寒冷，觉得应该给自己添加几件保暖的衣服了，于是想到了天猫，搜寻了一番，觉得南极人的保暖内衣还是不错的。到低怎么选择这么多的衣服呢？我一般选择按销量排序，毕竟销量也能侧面反映商品的受欢迎度和口碑状况，所以我来到了这个页面...
View ArticleClik here to view.
python-函数
函数是组织好的，可重复使用的，用来实现单一，或相关联功能的代码段。 函数能提高应用的模块性，和代码的重复利用率。你已经知道python提供了许多内建函数，比如print()。但你也可以自己创建函数，这被叫做用户自定义函数。 语法 def functionname( parameters ): function_suite return [expression] 实例 def printme(...
View ArticleClik here to view.
Python 云计算框架 Openstack 源码分析之 RabbitMQ（一）
在Openstack中，各个组件内部使用消息队列进行通信，其中，RabbitMQ是常用的一种开源消息代理软件。这里作一个简要介绍。 RabbitMQ介绍 RabbitMQ实现了高级消息队列协议（AMQP）。 AMQP AMQP是一个定义了应用之间消息传送协议的开放标准. AMQP旨在解决在两个应用之间传送消息存在的以下问题: 网络是不可靠的 -> 消息需要保存后再转发并有出错处理机制...
View ArticleClik here to view.
From Hairstylist to Mobile & Python Engineer: Brian’s Story
Brianhad been a professional hairstylist for 10 years when he joined Treehouse. He had first started programming in his teens and dabbled with freelance projects. However, when he’d chosen to focus on...
View ArticleClik here to view.
python之路:Day01 --- Python基础1
本节内容 1.python介绍 2.发展史 3.变量 4.用户输入 5.表达式 if...else语句 6.表达式 for 循环 7.表达式 while 循环 8.模块初始 9.数据类型初始 10.数据运算 一、Pyhon介绍 一、编译型与解释型语言 编译型语言是将源程序翻译成可执行的目标代码，翻译与执行是分开的；而解释型语言是对源程序的翻译与执行一次性完成，不生成可存储的目标代...
View Article