More precisely, I crawled 250,113,669 pages for just under 580 dollars in 39 hours and 25 minutes, using 20 Amazon EC2 machine instances.
I carried out this project because (among several other reasons) I wanted to understand what resources are required to crawl a small but non-trivial fraction of the web. In this post I describe some details of what I did. Of course, there’s nothing especially new: I wrote a vanilla (distributed) crawler, mostly to teach myself something about crawling and distributed computing. Still, I learned some lessons that may be of interest to a few others, and so in this post I describe what I did. …
You are given an array of distinct integers arr
and an array of integer arrays pieces
, where the integers in pieces
are distinct. Your goal is to form arr
by concatenating the arrays in pieces
in any order. However, you are not allowed to reorder the integers in each array pieces[i]
.
Return true
if it is possible to form the array arr
from pieces
. Otherwise, return false
.
Example 1:
Input: arr = [85], pieces = [[85]]
Output: true
Example 2:
Input: arr = [15,88], pieces = [[88],[15]]
Output: true
Explanation: Concatenate [15] then [88]
Example 3:
Input: arr = [49,18,16], pieces = [[16,18,49]]
Output: false
Explanation: Even though the numbers match, we cannot reorder pieces[0]. …
We have n
cities labeled from 1
to n
. Two different cities with labels x
and y
are directly connected by a bidirectional road if and only if x
and y
share a common divisor strictly greater than some threshold
. More formally, cities with labels x
and y
have a road between them if there exists an integer z
such that all of the following are true:
x % z == 0
,y % z == 0
, andz > threshold
.Given the two integers, n
and threshold
, and an array of queries
, you must determine for each queries[i] = [ai, bi]
if cities ai
and bi
are connected (i.e. …
The general answer is to 1) maintain a queue of 10,000 jobs, each one is to query a machine’s status 2) have a bunch of worker machines run in parallel, retrieving and executing one status querying job at a time.
Following are the steps of getting deeper in this question and ultimately reach a satisfying state.
A pitfall when the interviewer presents this question is that, he won’t tell you how many machines you can use to do such an operation. Thus, the first impression an inteviewee got is it can be a dead simple a loop.
List<Status> getStatus(List<Host> hosts) {
List<Status> status = Lists.newArrayList();
hosts.forEach(h -> {
status.add(h.getStatus()); …
What happens when we start a program and pass parameter?
We can start a program and pass parameter as a command line argument. We can read it those passing parameter easily by rust provided std::env module. Here is an example:
Inspiration: One of my friend recently called and told me he is facing a problem. He is trying to set up some environment variable in the server. He sshed into the machine and set up an environment variable. He does not see those environment variables after closing the session and connected back again. More precisely, app server does not see those env variable. I am writing these to explain what happens behind the scene and why does it happen.
What happens when you ssh into a machine?
Basically, you get the access to a shell of the machine securely.
What is shell? …
Sockets are a method of IPC that allows data to be exchanged between application in same host or remote host. In the telephonic world, if i want to communicate with my friend, I need a telephone and my friend needs a telephone as well. In the telephonic world, “telephone” is the socket.
In typical client-server application, applications communicate by using sockets.
How to create a socket, in general, we depend on the system call for that. …
For all the people frustrated by having to use to_string()
to get programs to compile this post is for you. For those not quite understanding why Rust has two string types String
and &str
, I hope to shed a little light on the matter.
I want to discuss how to build interfaces that accept strings. I am an avid hypermedia fan and am obsessed about designing interfaces that are easy to use. Let’s start with a method that accepts a String. Our search hints that std::string::String
is a good choice here.
fn print_me(msg: String) {
println!("the …
There are 3n piles of coins of varying size, you and your friends will take piles of coins as follows:
Given an array of integers piles
where piles[i]
is the number of coins in the ith
pile.
Return the maximum number of coins which you can have. …
About