事情是这样的,我每周末都会到处寻找有意思的活动参加,豆瓣活动是我的一个重要寻找来源之一。但是那个链接每次都记不住,正好搭建了博客,想直接挂在自己页面上,就方便多了。说干就干,一个iframe 就搞定的事情,分分钟解决。
可能是因为我平时都是在手机上看,豆瓣活动页面是有专门的手机版的,总感觉这个很难看,想要手机版。一开始以为只要改了`iframe`的宽度,豆瓣就会直接识别成手机版,然而事实并不是这样。查看请求发现,豆瓣是服务端渲染的,也就是说,页面到浏览器之前,就已经判定为pc 版或者手机版了。看来是通过请求头里的user-agent 字段来判断的。那就修改请求的ua 呗。查了一圈,iframe 无法修改ua……
这就比较麻烦了,得通过服务端转发一下。其实这个需求挺通用的,一直想好好写一个转发服务来着。不过最近还是比较忙,先不搞了,简单用nginx 配置一下转发吧。
location /douban/ { proxy_pass https://beijing.douban.com/; proxy_set_header Referer https://beijing.douban.com; proxy_set_header User-Agent "Mozilla/5.0 (Linux; Android 6.0; NEM-AL10 Build/HONORNEM-AL10; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/57.0.2987.132 MQQBrowser/6.2 TBS/043906 Mobile Safari/537.36 MicroMessenger/6.6.1.1220(0x26060133) NetType/WIFI Language/zh_CN"; }
就这样5行代码搞定。跑起来一看,还可以,但是一点击链接,发现不是那么回事,链接里地址还是 https://beijing.douban.com/,跳转一次之后,页面又变成pc 版了,蛋疼。要是能把html 里的链接也替换掉,不就搞定了吗?
查了一下资料,几行代码搞定。不过这几行代码可是花了我好多时间。
location /douban/beijing/ { proxy_pass https://beijing.douban.com/; # 反向代理 sub_filter_once off; # 全局替换 sub_filter 'beijing.douban.com' 'www.xxx.cn/douban/beijing'; #字符串替换 proxy_set_header Accept-Encoding ""; # 禁用gzip才能替换成功 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Real-IP $remote_addr; proxy_set_header Referer https://beijing.douban.com; proxy_set_header User-Agent "Mozilla/5.0 (Linux; Android 6.0; NEM-AL10 Build/HONORNEM-AL10; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/57.0.2987.132 MQQBrowser/6.2 TBS/043906 Mobile Safari/537.36 MicroMessenger/6.6.1.1220(0x26060133) NetType/WIFI Language/zh_CN"; }
点击详情页的时候,坑爹的事情又发生了,详情页的链接居然是www.douban.com,当然是还需要再加点东西了。为了不多写代码,想到了用变量。
location ~* /douban/(.+)/(.*) { proxy_pass https://$1.douban.com/$2?$args; resolver 8.8.8.8;//增加DNS解析地扯 sub_filter 'beijing.douban.com' 'www.xxx.cn/douban/beijing'; sub_filter 'www.douban.com' 'www.xxx.cn/douban/www'; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Real-IP $remote_addr; proxy_set_header Referer https://beijing.douban.com; proxy_set_header User-Agent "Mozilla/5.0 (Linux; Android 6.0; NEM-AL10 Build/HONORNEM-AL10; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/57.0.2987.132 MQQBrowser/6.2 TBS/043906 Mobile Safari/537.36 MicroMessenger/6.6.1.1220(0x26060133) NetType/WIFI Language/zh_CN"; }
然而这一次我死活没成功,搞了一晚上,一直报一个错误:
1 * beijing could not be resolved (3: Host not found)
查资料说要增加 resolver 解析地址,我加了,但是还是没成功.
➜ ~ dig @8.8.8.8 beijing.douban.com
; <<>> DiG 9.10.3-P4-Ubuntu <<>> @8.8.8.8 beijing.douban.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 40740
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;beijing.douban.com. IN A
;; ANSWER SECTION:
beijing.douban.com. 299 IN A 154.8.131.172
beijing.douban.com. 299 IN A 154.8.131.165
beijing.douban.com. 299 IN A 154.8.131.171
;; Query time: 76 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Jan 17 02:02:41 CST 2020
;; MSG SIZE rcvd: 95
我怀疑是nginx 把域名解析成beijing 了。
➜ ~ dig @8.8.8.8 beijing
; <<>> DiG 9.10.3-P4-Ubuntu <<>> @8.8.8.8 beijing
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 23333
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;beijing. IN A
;; AUTHORITY SECTION:
. 86398 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2020011601 1800 900 604800 86400
;; Query time: 39 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Jan 17 02:04:09 CST 2020
;; MSG SIZE rcvd: 111
重启了几十次调试,还是没成功。实在是太晚了,明天还得上班,先用笨办法顶上吧,有谁要是有解决方案,麻烦指教一下~
location /douban/beijing/ { proxy_pass https://beijing.douban.com/; # 反向代理 sub_filter_once off; # 全局替换 sub_filter 'beijing.douban.com' 'www.redream.cn/douban/beijing'; #字符串替换 sub_filter 'www.douban.com' 'www.redream.cn/douban/www'; proxy_set_header Accept-Encoding ""; # 禁用gzip才能替换成功 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Real-IP $remote_addr; proxy_set_header Referer https://beijing.douban.com; proxy_set_header User-Agent "Mozilla/5.0 (Linux; Android 6.0; NEM-AL10 Build/HONORNEM-AL10; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/57.0.2987.132 MQQBrowser/6.2 TBS/043906 Mobile Safari/537.36 MicroMessenger/6.6.1.1220(0x26060133) NetType/WIFI Language/zh_CN"; } location /douban/www/ { proxy_pass https://www.douban.com/; sub_filter_once off; sub_filter 'beijing.douban.com' 'www.redream.cn/douban/beijing'; sub_filter 'www.douban.com' 'www.redream.cn/douban/www'; proxy_set_header Accept-Encoding ""; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Real-IP $remote_addr; proxy_set_header Referer https://beijing.douban.com; proxy_set_header User-Agent "Mozilla/5.0 (Linux; Android 6.0; NEM-AL10 Build/HONORNEM-AL10; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/57.0.2987.132 MQQBrowser/6.2 TBS/043906 Mobile Safari/537.36 MicroMessenger/6.6.1.1220(0x26060133) NetType/WIFI Language/zh_CN"; }
更新一下一些新的方法
- 使用include
server { location /first/location/ { include shared.conf; } location /second/location/ { include shared.conf; } }
- 使用@引用
error_page 418 = @common_location; location /first/location/ { return 418; } location /second/location/ { return 418; } location @common_location { # The common configuration... }
- 使用嵌套location
location /specialpages/ { # some config location /specialpages/static/ { try_files $uri $uri/ =404; } location /specialpages/dynamic/ { proxy_pass http://127.0.0.1; } }
放上效果地址:北京周末去哪玩
参考文档:
* https://juejin.im/post/5bd7a6046fb9a05d2c43f8c7#heading-1
* https://moonbingbing.gitbooks.io/openresty-best-practices/content/ngx/nginx_brief.html