蕲春人 - 软件开发

简单就是美.

Shell学习小记录

最近为了一些自动化的任务,要写一些脚本。可以使用Ruby脚本、Ruby的rake\thor等等,我为了简洁和性能,我了解了一下Shell,发现用Shell来做这个事情更合适。性能,命令行,管道,丰富的现有工具,基于Linux本身,Shell真是一个好东西。以前认为用Ruby来做这个事情是一个不错的选择,现在知道了,那是因为相比起Shell来,更熟悉Ruby,程序员总是喜欢自己熟悉的领域,而排斥自己不熟悉的领域。其实多了解一下其它方面的,更利用自己工作的开展,提高工作效率。

我随便总结一下几个知识点

字符串

声明一个字符串变量后,使用的时候,在变量名前面加一个$符号才能将其值取出来

1
2
DATA_FILE=data.tar.gz
echo $DATA_FILE

字符串拼接

1
2
3
DATA_DIR=/Users/caojinhua/code/
DATA_FILE=data.tar.gz
DATA_PATH=$DATA_DIR""$DATA_FILE

将命令执行的结果保存在变量中

1
sha1=`ls -al`

if语句结构

if语句条件测试命令:

1
2
3
4
5
6
7
[ -d DIR ]   如果DIR存在并且是一个目录则为真
[ -f FILE ]   如果FILE存在且是一个普通文件则为真
[ -z STRING ] 如果STRING的长度为零则为真
[ -n STRING ] 如果STRING的长度非零则为真
[ STRING1 = STRING2 ] 如果两个字符串相同则为真
[ STRING1 != STRING2 ]    如果字符串不相同则为真
[ ARG1 OP ARG2 ]

ARG1和ARG2应该是整数或者取值为整数的变量,OP是-eq(等于)-ne(不等于)-lt(小于)-le(小于等于)-gt(大于)-ge(大于等于)之中的一个

之前弄错好几次,中括号前后的空格不能少。

if语句的结构

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
if [ xxx ]
then
fi

if [ xxx ]; then

else

fi

if [ xxx ]; then

elif [ ! xxx ]; then

fi

时间格式化

1
2
3
4
date  +%Y%m%d

a=`date +%Y%m%d`
echo $a

参考资料

Shell脚本语法

Fix Problem Annotate Is Broken With Rails 3.1.1

今天在做一个小工具,使用最新的Rails版本3.1.1, 在使用annotate(2.4.0)这个gem的时候出错了:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
caojinhua:tts_cacher caojinhua$ annotate
/Users/caojinhua/.rvm/gems/ruby-1.9.2-p180/gems/activerecord-3.1.1/lib/active_record/railties/databases.rake:3:in `<top (required)>': undefined method `namespace' for main:Object (NoMethodError)
  from /Users/caojinhua/.rvm/gems/ruby-1.9.2-p180/gems/activerecord-3.1.1/lib/active_record/railtie.rb:26:in `load'
  from /Users/caojinhua/.rvm/gems/ruby-1.9.2-p180/gems/activerecord-3.1.1/lib/active_record/railtie.rb:26:in `block in <class:Railtie>'
  from /Users/caojinhua/.rvm/gems/ruby-1.9.2-p180/gems/railties-3.1.1/lib/rails/railtie.rb:183:in `call'
  from /Users/caojinhua/.rvm/gems/ruby-1.9.2-p180/gems/railties-3.1.1/lib/rails/railtie.rb:183:in `block in load_tasks'
  from /Users/caojinhua/.rvm/gems/ruby-1.9.2-p180/gems/railties-3.1.1/lib/rails/railtie.rb:183:in `each'
  from /Users/caojinhua/.rvm/gems/ruby-1.9.2-p180/gems/railties-3.1.1/lib/rails/railtie.rb:183:in `load_tasks'
  from /Users/caojinhua/.rvm/gems/ruby-1.9.2-p180/gems/railties-3.1.1/lib/rails/engine.rb:396:in `block in load_tasks'
  from /Users/caojinhua/.rvm/gems/ruby-1.9.2-p180/gems/railties-3.1.1/lib/rails/application/railties.rb:8:in `each'
  from /Users/caojinhua/.rvm/gems/ruby-1.9.2-p180/gems/railties-3.1.1/lib/rails/application/railties.rb:8:in `all'
  from /Users/caojinhua/.rvm/gems/ruby-1.9.2-p180/gems/railties-3.1.1/lib/rails/engine.rb:396:in `load_tasks'
  from /Users/caojinhua/.rvm/gems/ruby-1.9.2-p180/gems/railties-3.1.1/lib/rails/application.rb:103:in `load_tasks'
  from /Users/caojinhua/.rvm/gems/ruby-1.9.2-p180/gems/railties-3.1.1/lib/rails/railtie/configurable.rb:30:in `method_missing'
  from Rakefile:7:in `<top (required)>'
  from /Users/caojinhua/.rvm/gems/ruby-1.9.2-p180/gems/annotate-2.4.0/lib/annotate.rb:17:in `load'
  from /Users/caojinhua/.rvm/gems/ruby-1.9.2-p180/gems/annotate-2.4.0/lib/annotate.rb:17:in `load_tasks'
  from /Users/caojinhua/.rvm/gems/ruby-1.9.2-p180/gems/annotate-2.4.0/bin/annotate:66:in `<top (required)>'
  from /Users/caojinhua/.rvm/gems/ruby-1.9.2-p180/bin/annotate:19:in `load'
  from /Users/caojinhua/.rvm/gems/ruby-1.9.2-p180/bin/annotate:19:in `<main>'

然后在stackoverflow上找到了解决方法,就是使用最新的annotate.

Gemfile
1
gem 'annotate', :git => 'git://github.com/ctran/annotate_models.git'

Resources:

annotate-gem-and-rails-3-1

网页截屏的方法

曾经很想有这样一个app, 它可以将微博上用户的微博用图片的形式自动保存起来,留此存照。前几个月的那段时间,微博上很 网页截屏的基本原理就是通过取得webkit渲染(render)的数据来生成图片的,我经过一段时间研究,找到了两个方法来解决这个问题。

一个工具叫phantomjs,另一个工具叫cutycapt

两个工具都不错,个人比较喜欢使用cutycapt这个工具,它是直接提供一个命令行来生成网页截图的,而前者是通过javascript来调用底层webkit接品(page.render方法)来实现的,两者的侧重点不一样。 并且cutycapt是将整个网截下来,phantomjs是将浏览器当前视区的一屏截下来。

Cutycapt的用法如下

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
qichunren@qichunren-desktop:~/github/cutycapt/CutyCapt$ ./CutyCapt 
 -----------------------------------------------------------------------------
 Usage: CutyCapt --url=http://www.example.org/ --out=localfile.png            
 -----------------------------------------------------------------------------
  --help                         Print this help page and exit                
  --url=<url>                    The URL to capture (http:...|file:...|...)   
  --out=<path>                   The target file (.png|pdf|ps|svg|jpeg|...)   
  --out-format=<f>               Like extension in --out, overrides heuristic 
  --min-width=<int>              Minimal width for the image (default: 800)   
  --min-height=<int>             Minimal height for the image (default: 600)  
  --max-wait=<ms>                Don't wait more than (default: 90000, inf: 0)
  --delay=<ms>                   After successful load, wait (default: 0)     
  --user-style-path=<path>       Location of user style sheet file, if any    
  --user-style-string=<css>      User style rules specified as text           
  --header=<name>:<value>        request header; repeatable; some can't be set
  --method=<get|post|put>        Specifies the request method (default: get)  
  --body-string=<string>         Unencoded request body (default: none)       
  --body-base64=<base64>         Base64-encoded request body (default: none)  
  --app-name=<name>              appName used in User-Agent; default is none  
  --app-version=<version>        appVers used in User-Agent; default is none  
  --user-agent=<string>          Override the User-Agent header Qt would set  
  --javascript=<on|off>          JavaScript execution (default: on)           
  --java=<on|off>                Java execution (default: unknown)            
  --plugins=<on|off>             Plugin execution (default: unknown)          
  --private-browsing=<on|off>    Private browsing (default: unknown)          
  --auto-load-images=<on|off>    Automatic image loading (default: on)        
  --js-can-open-windows=<on|off> Script can open windows? (default: unknown)  
  --js-can-access-clipboard=<on|off> Script clipboard privs (default: unknown)
  --print-backgrounds=<on|off>   Backgrounds in PDF/PS output (default: off)  
  --zoom-factor=<float>          Page zoom factor (default: no zooming)       
  --zoom-text-only=<on|off>      Whether to zoom only the text (default: off) 
  --http-proxy=<url>             Address for HTTP proxy server (default: none)
 -----------------------------------------------------------------------------
  <f> is svg,ps,pdf,itext,html,rtree,png,jpeg,mng,tiff,gif,bmp,ppm,xbm,xpm    
 -----------------------------------------------------------------------------
 http://cutycapt.sf.net - (c) 2003-2010 Bjoern Hoehrmann - bjoern@hoehrmann.de
qichunren@qichunren-desktop:~/github/cutycapt/CutyCapt$

phantomjs截屏的用法如下:

1
2
3
4
5
6
7
8
9
qichunren@qichunren-desktop:~/github$ cd phantomjs/
qichunren@qichunren-desktop:~/github/phantomjs$ ls
bin         ChangeLog  ff.png     iteye.png    Makefile       python     src
capture.js  examples   hello.png  LICENSE.BSD  phantomjs.pro  README.md           
qichunren@qichunren-desktop:~/github/phantomjs$ ./bin/phantomjs examples/rasterize.js 
Usage: rasterize.js URL filename [paperwidth*paperheight|paperformat]
  paper (pdf output) examples: "5in*7.5in", "10cm*20cm", "A4", "Letter"
qichunren@qichunren-desktop:~/github/phantomjs$ ./bin/phantomjs examples/rasterize.js http://www.iteye.com iteye.png
qichunren@qichunren-desktop:~/github/phantomjs$

使用Jruby来部署Rails应用

为了保护最近做的产品的源代码,需要将项目中的源代码进行保护起来。我目前了解到的方案有以下两种:

  1. 使用代码混淆工具

  2. 使用JRuby将Ruby代码编译成java字节码文件(.class)


第一种方案,有一个名为ruby encoder的产品,我试用了一下,发现太重量级了,我个人只是一个可以将代码混淆一下的小工具而已,而ruby encoder有自己的运行加载机制,源代码二次编码,基于域名可以设置产品过期失效时间等等一系列功能,我不需要这些功能,另外它不是免费的,所以我没有采用这个方案。

第二种方安装就是使用JRuby。整体思路就是将Ruby项目的代码编译成java字节码文件,然后运行于Java环境中。


将项目中的ruby文件编译成java的class文件不是一件容易的事情,所幸有一个名为warbler的gem可以帮助我们搞定这一切,它可以将项目打包(.war),同时可以将ruby代码编译成class文件。然后你将生成好的.war文件放进JAVA应用服务器的应用目录中,如Tomcat的webapps中就可以了。

warbler提供若干个任务可供使用:

1
2
3
4
5
6
7
8
9
10
qichunren@qichunren-desktop:~/code/ntdeck$ warble -T
warble compiled    # Feature: precompile all Ruby files
warble config      # Generate a configuration file to customize your archive
warble executable  # Feature: make an executable archive
warble gemjar      # Feature: package gem repository inside a jar
warble pluginize   # Install Warbler tasks in your Rails application
warble version     # Display version of Warbler
warble war         # Create the project war file
warble war:clean   # Remove the project war file
warble war:debug   # Dump diagnostic information

平时最常用的就是warble war命令了,需要关注的是warble的配置文件,它的配置文件是通过warble config来生成的,在这个文件中有一系列的配置项可以设置,以下是我的配置文件:

config/warble.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
# Disable Rake-environment-task framework detection by uncommenting/setting to false
# Warbler.framework_detection = false

# Warbler web application assembly configuration file
Warbler::Config.new do |config|
  # Features: additional options controlling how the jar is built.
  # Currently the following features are supported:
  # - gemjar: package the gem repository in a jar file in WEB-INF/lib
  # - executable: embed a web server and make the war executable
  # - compiled: compile .rb files to .class files
  config.features = %w(executable compiled)

  # Application directories to be included in the webapp.
  config.dirs = %w(app config db lib log vendor tmp)

  # Additional files/directories to include, above those in config.dirs
  # config.includes = FileList["db"]

  # Additional files/directories to exclude
  # config.excludes = FileList["lib/tasks/*"]

  # Additional Java .jar files to include.  Note that if .jar files are placed
  # in lib (and not otherwise excluded) then they need not be mentioned here.
  # JRuby and JRuby-Rack are pre-loaded in this list.  Be sure to include your
  # own versions if you directly set the value
  # config.java_libs += FileList["lib/java/*.jar"]

  # Loose Java classes and miscellaneous files to be included.
  # config.java_classes = FileList["target/classes/**.*"]

  # One or more pathmaps defining how the java classes should be copied into
  # the archive. The example pathmap below accompanies the java_classes
  # configuration above. See http://rake.rubyforge.org/classes/String.html#M000017
  # for details of how to specify a pathmap.
  # config.pathmaps.java_classes << "%{target/classes/,}p"

  # Bundler support is built-in. If Warbler finds a Gemfile in the
  # project directory, it will be used to collect the gems to bundle
  # in your application. If you wish to explicitly disable this
  # functionality, uncomment here.
  # config.bundler = false

  # An array of Bundler groups to avoid including in the war file.
  # Defaults to ["development", "test"].
  # config.bundle_without = []

  # Other gems to be included. If you don't use Bundler or a gemspec
  # file, you need to tell Warbler which gems your application needs
  # so that they can be packaged in the archive.
  # For Rails applications, the Rails gems are included by default
  # unless the vendor/rails directory is present.
  # config.gems += ["activerecord-jdbcmysql-adapter", "jruby-openssl"]
  # config.gems << "tzinfo"

  # Uncomment this if you don't want to package rails gem.
  # config.gems -= ["rails"]

  # The most recent versions of gems are used.
  # You can specify versions of gems by using a hash assignment:
  # config.gems["rails"] = "2.3.10"

  # You can also use regexps or Gem::Dependency objects for flexibility or
  # finer-grained control.
  # config.gems << /^merb-/
  # config.gems << Gem::Dependency.new("merb-core", "= 0.9.3")

  # Include gem dependencies not mentioned specifically. Default is
  # true, uncomment to turn off.
  # config.gem_dependencies = false

  # Array of regular expressions matching relative paths in gems to be
  # excluded from the war. Defaults to empty, but you can set it like
  # below, which excludes test files.
  # config.gem_excludes = [/^(test|spec)\//]

  # Pathmaps for controlling how application files are copied into the archive
  # config.pathmaps.application = ["WEB-INF/%p"]

  # Name of the archive (without the extension). Defaults to the basename
  # of the project directory.
  config.jar_name = "ntdeck"

  # Name of the MANIFEST.MF template for the war file. Defaults to a simple
  # MANIFEST.MF that contains the version of Warbler used to create the war file.
  # config.manifest_file = "config/MANIFEST.MF"

  # When using the 'compiled' feature and specified, only these Ruby
  # files will be compiled. Default is to compile all \.rb files in
  # the application.
  # config.compiled_ruby_files = FileList['app/**/*.rb']
  compile_me = FileList[*config.dirs.map {|x| "#{x}/**/*.rb"}].exclude("config/compass.rb").exclude("lib/printer/*")
  config.compiled_ruby_files = compile_me

  # === War files only below here ===

  # Path to the pre-bundled gem directory inside the war file. Default
  # is 'WEB-INF/gems'. Specify path if gems are already bundled
  # before running Warbler. This also sets 'gem.path' inside web.xml.
  # config.gem_path = "WEB-INF/vendor/bundler_gems"

  # Files for WEB-INF directory (next to web.xml). This contains
  # web.xml by default. If there is an .erb-File it will be processed
  # with webxml-config. You may want to exclude this file via
  # config.excludes.
  # config.webinf_files += FileList["jboss-web.xml"]

  # Files to be included in the root of the webapp.  Note that files in public
  # will have the leading 'public/' part of the path stripped during staging.
  # config.public_html = FileList["public/**/*", "doc/**/*"]

  # Pathmaps for controlling how public HTML files are copied into the .war
  # config.pathmaps.public_html = ["%{public/,}p"]

  # Value of RAILS_ENV for the webapp -- default as shown below
  # config.webxml.rails.env = ENV['RAILS_ENV'] || 'production'

  # Application booter to use, one of :rack, :rails, or :merb (autodetected by default)
  # config.webxml.booter = :rails

  # Set JRuby to run in 1.9 mode.
  # config.webxml.jruby.compat.version = "1.9"

  # When using the :rack booter, "Rackup" script to use.
  # - For 'rackup.path', the value points to the location of the rackup
  # script in the web archive file. You need to make sure this file
  # gets included in the war, possibly by adding it to config.includes
  # or config.webinf_files above.
  # - For 'rackup', the rackup script you provide as an inline string
  #   is simply embedded in web.xml.
  # The script is evaluated in a Rack::Builder to load the application.
  # Examples:
  # config.webxml.rackup.path = 'WEB-INF/hello.ru'
  # config.webxml.rackup = %{require './lib/demo'; run Rack::Adapter::Camping.new(Demo)}
  # config.webxml.rackup = require 'cgi' && CGI::escapeHTML(File.read("config.ru"))

  # Control the pool of Rails runtimes. Leaving unspecified means
  # the pool will grow as needed to service requests. It is recommended
  # that you fix these values when running a production server!
  config.webxml.jruby.min.runtimes = 1
  config.webxml.jruby.max.runtimes = 1

  # JNDI data source name
  # config.webxml.jndi = 'jdbc/rails'
end

需要注意的是config.features = %w(executable compiled)配置中,其中的compiled就是可以将ruby代码编译成class代码的。

Set Proxy in Server Side to Get Crossing Domain Ajax Request

proxy_controller.rb
1
2
3
4
5
6
7
8
9
10
11
class ProxyController < ApplicationController

  # GET /proxy/:url
  def get_handle
    require 'open-uri'
    file = open(params[:url])
    contents = file.read
    render :text => contents
  end

end
in config/routes.rb
1
get "/proxy" => "proxy#get_handle"
javascript useage
1
$.get("/proxy?url=" + remote_url, function(data){

Install Mongo on Mac

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
caojinhuamatoMacBook-Pro:code caojinhua$ brew install mongodb
==> Downloading http://fastdl.mongodb.org/osx/mongodb-osx-x86_64-1.8.1.tgz
######################################################################## 100.0%
==> Caveats
If this is your first install, automatically load on login with:
    mkdir -p ~/Library/LaunchAgents
    cp /usr/local/Cellar/mongodb/1.8.1-x86_64/org.mongodb.mongod.plist ~/Library/LaunchAgents/
    launchctl load -w ~/Library/LaunchAgents/org.mongodb.mongod.plist

If this is an upgrade and you already have the org.mongodb.mongod.plist loaded:
    launchctl unload -w ~/Library/LaunchAgents/org.mongodb.mongod.plist
    cp /usr/local/Cellar/mongodb/1.8.1-x86_64/org.mongodb.mongod.plist ~/Library/LaunchAgents/
    launchctl load -w ~/Library/LaunchAgents/org.mongodb.mongod.plist

Or start it manually:
    mongod run --config /usr/local/Cellar/mongodb/1.8.1-x86_64/mongod.conf
MongoDB 1.8+ includes a feature for Write Ahead Logging (Journaling), which has been enabled by default.
This is not the default in production (Journaling is disabled); to disable journaling, use --nojournal.
==> Summary
/usr/local/Cellar/mongodb/1.8.1-x86_64: 16 files, 93M, built in 2 seconds

去掉textmate源代码中的隐形空格

有这个需要,主要是因为不想在git提交后的diff中无看到不有意思的diff显示。使用textmate的同学可以用这个工具来解决这个小问题。

安装方法

install step
1
2
3
4
5
cd ~/Library/Application\ Support/TextMate/Bundles/
git clone git://github.com/glennr/uber-glory-tmbundle.git Uber\ Glory.tmbundle
cd Uber\ Glory.tmbundle
git submodule update --init
osascript -e 'tell app "TextMate" to reload bundles'

使用Unicorn

Install

1
gem install unicorn

Then add gem ‘unicorn’ to Gemfile. bundle exec unicorn_rails to start rails app at 8080 port.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
caojinhua:qichunren.github.com caojinhua$ unicorn_rails --help
Usage: unicorn_rails [ruby options] [unicorn_rails options] [rackup config file]
Ruby options:
  -e, --eval LINE          evaluate a LINE of code
  -d, --debug              set debugging flags (set $DEBUG to true)
  -w, --warn               turn warnings on for your script
  -I, --include PATH       specify $LOAD_PATH (may be used more than once)
  -r, --require LIBRARY    require the library, before executing your script
unicorn_rails options:
  -o, --host HOST          listen on HOST (default: 0.0.0.0)
  -p, --port PORT          use PORT (default: 8080)
  -E, --env RAILS_ENV      use RAILS_ENV for defaults (default: development)
  -D, --daemonize          run daemonized in the background
  -l {HOST:PORT|PATH},     listen on HOST:PORT or PATH
      --listen             this may be specified multiple times
                           (default: 0.0.0.0:8080)
  -c, --config-file FILE   Unicorn-specific config file

      --path PATH          Runs Rails app mounted at a specific path.
                           (default: /)
Common options:
  -h, --help               Show this message
  -v, --version            Show version

This is a smaple unicorn config file:

config/unicorn.conf.rb
1
2
3
4
5
6
7
worker_processes 2
working_directory "/www/temp/ntmenu2/current"
listen 3000, :tcp_nopush => true
timeout 30
pid "/www/temp/ntmenu2/current/tmp/pids/unicorn.pid"
stderr_path "/www/temp/unicorn.stderr.log"
stdout_path "/www/temp/unicorn.stdout.log"

使用Juggernaut

Juggernaut是基于Node.js的一个实时(Realtime)Web的解决方案。使用起来很方便。

安装方法

  1. 安装Node.js: brew install node
  2. 安装Redis: brew install redis
  3. 安装NPM: curl http://npmjs.org/install.sh | sh
  4. 安装Juggernaut: 这个会把Juggernaut安装到当前目录,所以我应该先进行项目的/vendor/third目录,然后执行 npm install juggernaut
1
2
3
4
5
6
usermatoMacBook-Pro:third qichunren$ npm install juggernaut
redis@0.5.11 ./node_modules/juggernaut/node_modules/redis
node-static-maccman@0.5.3 ./node_modules/juggernaut/node_modules/node-static-maccman
socket.io@0.6.17 ./node_modules/juggernaut/node_modules/socket.io
juggernaut@2.0.4 ./node_modules/juggernaut
usermatoMacBook-Pro:third qichunren$  
  1. 安装Juggernaut gem:
1
gem install juggernaut

使用方法

我们在自己的项目中只需要引入http://localhost:8080/application.js 这个js文件即可。 然后在页面中可以这样接收服务器端的消息:

1
2
3
4
5
6
<script type="text/javascript" charset="utf-8">
  var jug = new Juggernaut;
  jug.subscribe("channel1", function(data){
    console.log("Got data: " + data);
  });
</script>

服务器端直接发消息:

1
2
require "juggernaut"
Juggernaut.publish("channel1", "Some data")

另外在启动了Juggernaut后,它默认在8080端口上有一个Helloword的应用,可以了解一下。