python将gfwlist转换成dnsmasq规则

post by rocdk890 / 2016-3-14 14:41 Monday linux技术
  当使用openwrt时,大家选择fq工具一定是ss,如果你是使用的石像鬼的固件的话,那可以不用看这篇文章了,如果是使用官方默认openwrt或者明月的openwrt的话,那么下面这个东西会为你带来惊喜,其实也就是用python转换gfwlist为dnsmasq规则.
  需要的软件:python 2.6

脚本内容:
cat dnsmasq-gfwlist.py
#!/usr/bin/env python 
#coding=utf-8
# 
# Generate a list of dnsmasq rules with ipset for gfwlist
# 
# Copyright (C) 2014 http://www.shuyz.com  
# Ref https://code.google.com/p/autoproxy-gfwlist/wiki/Rules   
 
import urllib2
import re
import os
import datetime
import base64
import shutil
 
mydnsip = '127.0.0.1'
mydnsport = '1053'

# the url of gfwlist
##baseurl = 'http://dl.aenes.com/gfwlist.txt' (其实这个也可以用,具体选那个看你个人的需求.)
baseurl = 'https://autoproxy-gfwlist.googlecode.com/svn/trunk/gfwlist.txt'
# match comments/title/whitelist/ip address
comment_pattern = '^\!|\[|^@@|^\d+\.\d+\.\d+\.\d+'
domain_pattern = '([\w\-\_]+\.[\w\.\-\_]+)[\/\*]*'
tmpfile = 'gfwlisttmp'
# do not write to router internal flash directly
outfile = 'gfwlist.conf'
rulesfile = 'gfwlist.conf'
 
fs =  file(outfile, 'w')
fs.write('# gfw list ipset rules for dnsmasq\n')
fs.write('# updated on ' + datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S") + '\n')
fs.write('#\n')
 
print 'fetching list...'
content = urllib2.urlopen(baseurl, timeout=15).read().decode('base64')

# write the decoded content to file then read line by line
tfs = open(tmpfile, 'w')
tfs.write(content)
tfs.close()
tfs = open(tmpfile, 'r')

print 'page content fetched, analysis...'

# remember all blocked domains, in case of duplicate records
domainlist = []

for line in tfs.readlines():   
    if re.findall(comment_pattern, line):
        print 'this is a comment line: ' + line
        #fs.write('#' + line)
    else:
        domain = re.findall(domain_pattern, line)
        if domain:
            try:
                found = domainlist.index(domain[0])
                print domain[0] + ' exists.'
            except ValueError:
                print 'saving ' + domain[0]
                domainlist.append(domain[0])
                fs.write('server=/.%s/%s#%s\n'%(domain[0],mydnsip,mydnsport))
                fs.write('ipset=/.%s/gfwlist\n'%domain[0])
        else:
            print 'no valid domain in this line: ' + line
                   
tfs.close()
fs.close();

#print 'moving generated file to dnsmasg directory'
#shutil.move(outfile, rulesfile)

#print 'restart dnsmasq...'
#print os.popen('/etc/init.d/dnsmasq restart').read()
 
print 'done!'

保存后,执行:
chmod +x dnsmasq-gfwlist.py
python dnsmasq-gfwlist.py

如下图
点击查看原图
当然光是这样是不行的,如果你没有使用ipset的话,还需要修改下gfwlist.conf里的内容:
sed -n 'n;p' gfwlist.conf > t.conf
修改后的内容如下:
点击查看原图
这样就可以丢进openwrt里去给dnsmasq解析用了.
ps:https://gist.github.com/lanceliao/85cd3fcf1303dba2498c
夜空- 本站版权
1、本站所有主题由该文章作者发表,该文章作者与夜空享有文章相关版权
2、其他单位或个人使用、转载或引用本文时必须同时征得该文章作者和夜空的同意
3、本帖部分内容转载自其它媒体,但并不代表本站赞同其观点和对其真实性负责
4、如本帖侵犯到任何版权问题,请立即告知本站,本站将及时予与删除并致以最深的歉意
5、原文链接:blog.slogra.com/post-643.html

标签: dnsmasq 规则 转换 python openwrt gfw gfwlist

评论: