stackexchange-favs-to-pinboard

Check-in [cb04e6d1bf]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Updates for StackExchange API and Ruby 2.X

- As a result of previous commit, discovered that Ruby open-uri now
seems to automatically decompress Gzip data so parsing of responses is
more simple now.
- Also, update the API version
- Use response.read instead of string as some responses are too long for
string and get written to a tempfile
- Ensure we are making no more than 30 requests/sec

Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | origin/master | trunk
Files: files | file ages | folders
SHA3-256: cb04e6d1bfe975b5323c3e7f7626221151c2a3d2d61f19c9994c106e93c0c4bd
User & Date: base@atomicules.co.uk 2015-04-06 10:14:18
Context
2015-04-06
10:21
:Merge branch 'master' of github.com:atomicules/stackexchange-favs-to-pinboard check-in: 2f9937b611 user: base@atomicules.co.uk tags: origin/master, trunk
10:14
Updates for StackExchange API and Ruby 2.X

- As a result of previous commit, discovered that Ruby open-uri now
seems to automatically decompress Gzip data so parsing of responses is
more simple now.
- Also, update the API version
- Use response.read instead of string as some responses are too long for
string and get written to a tempfile
- Ensure we are making no more than 30 requests/sec check-in: cb04e6d1bf user: base@atomicules.co.uk tags: origin/master, trunk

10:00
Improvements to Pinboard code. Now a class, uses json and logger

I used the Pinboard code from this script as a basis for some Jekyll
syndication code I'm putting together elsewhere. I then made
changes/improvements to it which I've decided to port back over here. I
guess eventually I'll end up making my own Pinboard gem. Maybe.

Some corresponding changes to the script were necessary such as command
line arguments for Pinboard username and token, and re-ordering of method
arguments. check-in: c8517a3a50 user: base@atomicules.co.uk tags: origin/master, trunk

Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to stackexchange-favs-to-pinboard.rb.

8
9
10
11
12
13
14
15

16
17
18
19
20

21
22
23

24
25
26

27
28

29
30
31
32
33
34
35
36

optparse = OptionParser.new do |opts|
	opts.on('-i', '--id ID', "Stackexchange ID") { |i| StackID = i }
	opts.on('-u', '--user USER', "Pinboard username") { |u| User = u }
	opts.on('-t', '--token TOKEN', "Pinboard API Token") { |t| Token = t }
end
optparse.parse!


def get_sites(id)
	response = open("https://api.stackexchange.com/2.1/users/#{id}/associated")
	parsed = parse(response)
end


def parse(response) 
	#From: Garth, http://stackoverflow.com/a/1366187/208793
	gz = Zlib::GzipReader.new(StringIO.new(response.string))

	parsed = JSON.parse(gz.read)
end


def get_favs(site, id)
	response = open("https://api.stackexchange.com/2.1/users/#{id}/favorites?order=desc&sort=activity&site=#{site}")

	parsed = parse(response)
end


class Pinboard
	@@logger = Logger.new(STDOUT)
	@@logger.level = Logger::INFO
	@@rate_limit = 3








>

|
|


>
|
<
<
>
|


>

|
>
|







8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

optparse = OptionParser.new do |opts|
	opts.on('-i', '--id ID', "Stackexchange ID") { |i| StackID = i }
	opts.on('-u', '--user USER', "Pinboard username") { |u| User = u }
	opts.on('-t', '--token TOKEN', "Pinboard API Token") { |t| Token = t }
end
optparse.parse!


def get_sites(id)
	response = open("https://api.stackexchange.com/2.2/users/#{id}/associated")
	parsed = parse(response.read)
end


def parse(response_string)


	#Seems like Ruby automatically handles gzip compression now. No need to decompress.
	parsed = JSON.parse(response_string)
end


def get_favs(site, id)
	response = open("https://api.stackexchange.com/2.2/users/#{id}/favorites?order=desc&sort=activity&site=#{site}")
	#Use response.read not .string as some returns are large enough to go to a temp file
	parsed = parse(response.read)
end


class Pinboard
	@@logger = Logger.new(STDOUT)
	@@logger.level = Logger::INFO
	@@rate_limit = 3
87
88
89
90
91
92
93


94
95
96
97
98
99
100
101
102


if defined?(StackID) and defined?(User) and defined?(Token)
	pb = Pinboard.new(User, Token)
	parsed = get_sites(StackID)
	parsed["items"].each do |site|
		favs = get_favs(site["site_url"].sub("http://", "").sub(".stackexchange", "").sub(".com", ""), site["user_id"])


		favs["items"].each do |fav|
			title = fav["title"]
			tags = fav["tags"]
			link = fav["link"]
			#Need to unescape so can re-escape in Pinboard code
			pb.add(link, CGI.unescape_html(title), nil, tags.join(", ")+", stackexchangefavs", "no")
		end
	end
end







>
>









90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107


if defined?(StackID) and defined?(User) and defined?(Token)
	pb = Pinboard.new(User, Token)
	parsed = get_sites(StackID)
	parsed["items"].each do |site|
		favs = get_favs(site["site_url"].sub("http://", "").sub(".stackexchange", "").sub(".com", ""), site["user_id"])
		#Don't make more than 30 requests per second (unlikely too anyway, what with Pinboard adding)
		sleep (1.0/30)
		favs["items"].each do |fav|
			title = fav["title"]
			tags = fav["tags"]
			link = fav["link"]
			#Need to unescape so can re-escape in Pinboard code
			pb.add(link, CGI.unescape_html(title), nil, tags.join(", ")+", stackexchangefavs", "no")
		end
	end
end